Sep 16 18:14:15.075200 ip-10-0-133-100 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 16 18:14:15.075212 ip-10-0-133-100 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 16 18:14:15.075219 ip-10-0-133-100 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 16 18:14:15.075383 ip-10-0-133-100 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 16 18:14:25.124034 ip-10-0-133-100 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 16 18:14:25.124051 ip-10-0-133-100 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 23bb9c9479eb43d7a2fa431752ca8ba6 --
Sep 16 18:16:18.130378 ip-10-0-133-100 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 18:16:18.525138 ip-10-0-133-100 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:16:18.525138 ip-10-0-133-100 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 18:16:18.525138 ip-10-0-133-100 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:16:18.525138 ip-10-0-133-100 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 18:16:18.525684 ip-10-0-133-100 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:16:18.527142 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.527073    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 18:16:18.532186 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532173    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:18.532186 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532186    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532191    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532196    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532199    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532202    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532205    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532207    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532210    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532213    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532216    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532218    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532221    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532224    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532227    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532229    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532232    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532234    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532236    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532239    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532247    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:18.532252 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532249    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532252    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532255    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532258    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532260    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532263    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532266    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532268    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532271    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532273    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532276    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532278    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532281    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532283    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532286    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532288    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532291    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532293    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532296    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532299    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:18.532709 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532301    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532303    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532306    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532308    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532311    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532313    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532315    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532319    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532322    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532324    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532327    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532329    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532332    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532334    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532337    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532340    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532343    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532345    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532349    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532352    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:18.533237 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532356    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532360    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532363    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532366    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532369    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532372    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532375    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532377    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532379    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532382    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532384    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532387    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532390    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532392    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532395    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532398    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532400    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532403    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532407    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:18.533721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532409    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532419    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532422    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532425    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532427    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532430    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532433    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532441    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532444    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532447    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532449    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532452    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532455    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532457    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532460    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532462    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532465    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532467    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:18.534183 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.532470    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534280    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534288    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534292    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534295    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534299    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534302    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534305    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534308    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534311    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534313    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534316    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534319    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534321    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534324    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534326    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534329    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534332    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534334    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:18.534648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534337    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534341    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534345    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534348    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534351    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534354    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534357    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534360    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534364    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534366    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534369    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534371    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534374    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534376    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534380    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534383    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534386    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534389    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534392    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534394    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:18.535119 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534397    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534401    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534405    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534408    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534411    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534414    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534417    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534420    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534423    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534425    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534429    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534431    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534434    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534436    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534439    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534441    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534444    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534447    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534449    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534452    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:18.535594 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534454    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534457    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534460    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534462    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534465    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534468    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534471    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534473    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534476    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534479    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534482    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534484    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534487    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534489    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534492    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534494    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534497    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534499    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534502    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:18.536078 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534504    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534507    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534510    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534513    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534516    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534518    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534521    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534524    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534526    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534529    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534532    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534534    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534537    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534540    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534542    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534545    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534547    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534550    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534553    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534556    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:18.536529 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534558    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534561    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534925    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534930    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534933    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534936    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534938    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534941    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534944    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534947    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534949    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534952    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534954    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534957    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534959    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534962    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534970    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534972    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534975    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:18.537020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534977    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534980    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534988    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534991    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534993    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534996    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.534999    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535001    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535004    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535006    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535009    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535011    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535014    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535017    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535019    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535021    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535024    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535026    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535029    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535031    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:18.537476 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535033    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535036    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535038    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535041    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535043    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535046    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535048    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535051    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535053    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535056    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535059    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535061    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535064    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535067    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535069    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535085    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535088    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535092    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535095    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535097    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:18.537975 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535100    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535103    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535105    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535108    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535111    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535113    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535116    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535118    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535122    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535127    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535130    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535133    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535135    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535138    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535140    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535143    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535145    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535148    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535150    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:18.538456 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535154    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535157    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535161    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535164    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535167    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535170    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535172    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535179    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535181    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535184    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535187    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535190    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535192    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535195    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535198    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535201    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535203    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535206    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535209    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535212    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:18.538919 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535215    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535218    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535221    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535296    2562 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535346    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535355    2562 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535359    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535363    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535367    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535370    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535374    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535377    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535380    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535384    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535387    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535390    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535393    2562 flags.go:64] FLAG: --cgroup-root=""
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535396    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535399    2562 flags.go:64] FLAG: --client-ca-file=""
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535404    2562 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535407    2562 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535412    2562 flags.go:64] FLAG: --cluster-domain=""
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535415    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535418    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 18:16:18.539429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535421    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535424    2562 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535428    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535431    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535434    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535437    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535440    2562 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535444    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535447    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535450    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535453    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535457    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535460    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535463    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535466    2562 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535468    2562 flags.go:64] FLAG: --enable-server="true"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535471    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535475    2562 flags.go:64] FLAG: --event-burst="100"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535478    2562 flags.go:64] FLAG: --event-qps="50"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535481    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535484    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535487    2562 flags.go:64] FLAG: --eviction-hard=""
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535490    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535493    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535496    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 18:16:18.539998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535499    2562 flags.go:64] FLAG: --eviction-soft=""
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535502    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535506    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535509    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535511    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535514    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535517    2562 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535520    2562 flags.go:64] FLAG: --feature-gates=""
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535524    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535526    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535529    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535532    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535535    2562 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535538    2562 flags.go:64] FLAG: --help="false"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535540    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-133-100.ec2.internal"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535543    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535546    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535549    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535552    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535555    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535558    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535560    2562 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535563    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 18:16:18.540593 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535566    2562 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535568    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535571    2562 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535574    2562 flags.go:64] FLAG: --kube-reserved=""
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535576    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535579    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535582    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535584    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535587    2562 flags.go:64] FLAG: --lock-file=""
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535590    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535592    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535597    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535602    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535605    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535608    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535611    2562 flags.go:64] FLAG: --logging-format="text"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535614    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535619    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535622    2562 flags.go:64] FLAG: --manifest-url=""
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535625    2562 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535629    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535632    2562 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535636    2562 flags.go:64] FLAG: --max-pods="110"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535639    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535641    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 18:16:18.541176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535644    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535648    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535651    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535654    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535656    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535663    2562 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535665    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535668    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535671    2562 flags.go:64] FLAG: --pod-cidr=""
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535674    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535679    2562 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535681    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535684    2562 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535687    2562 flags.go:64] FLAG: --port="10250"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535690    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535693    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05541707830180def"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535696    2562 flags.go:64] FLAG: --qos-reserved=""
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535699    2562 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535703    2562 flags.go:64] FLAG: --register-node="true"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535706    2562 flags.go:64] FLAG: --register-with-taints=""
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535710    2562 flags.go:64] FLAG: --registry-burst="10"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535712    2562 flags.go:64] FLAG: --registry-qps="5"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535715    2562 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535718    2562 flags.go:64] FLAG: --reserved-memory=""
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535721    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 18:16:18.541774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535725    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535728    2562 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535731    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535733    2562 flags.go:64] FLAG: --runonce="false"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535736    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535739    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535742    2562 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535744    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535747    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535750    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535753    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535756    2562 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535759    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535761    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535764    2562 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535767    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535769    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535772    2562 flags.go:64] FLAG: --system-cgroups=""
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535775    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535779    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535782    2562 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535785    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535788    2562 flags.go:64] FLAG: --tls-min-version=""
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535791    2562 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535794    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 18:16:18.542386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535798    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535800    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535804    2562 flags.go:64] FLAG: --v="2"
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535807    2562 flags.go:64] FLAG: --version="false"
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535813    2562 flags.go:64] FLAG: --vmodule=""
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535817    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.535820    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535946    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535951    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535956    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535959    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535963    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535967    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535970    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535973    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535976    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535979    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535981    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535984    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535986    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:18.543074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535989    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535991    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535994    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535996    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.535999    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536001    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536004    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536007    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536009    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536011    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536014    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536016    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536020    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536023    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536025    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536028    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536031    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536033    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536036    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536039    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:18.543561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536042    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536044    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536047    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536049    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536052    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536054    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536057    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536060    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536062    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536065    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536067    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536070    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536072    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536075    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536077    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536080    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536082    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536086    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536090    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:18.544224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536093    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536096    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536099    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536101    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536103    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536107    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536110    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536113    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536115    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536118    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536120    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536123    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536126    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536129    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536132    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536134    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536137    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536139    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536142    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536144    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536146    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:18.544915 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536149    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536152    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536154    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536157    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536159    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536162    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536164    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536166    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536169    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536171    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536174    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536178    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536182    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536185    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536189    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536192    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536197    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536200    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536202    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536205    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:18.545421 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536207    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:18.545918 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536210    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:18.545918 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536213    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:18.545918 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536217    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:18.545918 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536219    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:18.545918 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.536222    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:18.545918 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.536906    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:16:18.546787 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.546770    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 18:16:18.546820 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.546787    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 18:16:18.547617 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.547607    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 18:16:18.547652 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.547617    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 18:16:18.547685 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547671    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:18.547685 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547676    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:18.547685 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547679    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:18.547685 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547682    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:18.547685 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547685    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547689    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547692    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547695    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547699    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547703    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547706    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547709    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547711    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547714    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547717    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547720    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547723    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547726    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547729    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547732    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547736    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547740    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547743    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:18.547808 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547746    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547748    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547751    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547753    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547756    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547758    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547761    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547763    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547766    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547768    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547771    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547773    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547776    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547778    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547781    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547784    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547787    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547790    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547792    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547795    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:18.548314 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547798    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547800    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547803    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547805    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547808    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547810    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547813    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547816    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547819    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547822    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547825    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547827    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547829    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547832    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547835    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547837    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547840    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547843    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547845    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547848    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:18.548791 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547851    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547853    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547870    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547875    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547878    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547880    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547883    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547885    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547888    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547891    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547894    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547896    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547900    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547902    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547904    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547907    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547911    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547914    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547916    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:18.549351 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547920    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547923    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547926    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547929    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547932    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547935    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547938    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547941    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547943    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547946    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547948    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547951    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547954    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547956    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547959    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547961    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:18.549799 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.547964    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.547969    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548074    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548078    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548082    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548085    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548088    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548091    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548094    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548096    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548100    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548103    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548107    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548111    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:18.550199 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548114    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548117    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548119    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548122    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548125    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548128    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548131    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548133    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548136    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548138    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548141    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548144    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548146    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548149    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548151    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548154    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548156    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548159    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548162    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548165    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548167    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:18.550543 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548170    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548172    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548175    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548177    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548180    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548183    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548186    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548188    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548191    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548193    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548197    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548200    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548203    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548206    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548208    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548211    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548213    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548216    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548219    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:18.551051 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548221    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548224    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548226    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548229    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548231    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548233    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548236    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548238    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548241    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548243    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548247    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548250    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548253    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548256    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548259    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548261    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548264    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548266    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548269    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:18.551517 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548272    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548276    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548279    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548281    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548284    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548286    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548289    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548292    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548294    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548297    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548299    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548302    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548304    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548307    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548309    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548311    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548314    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548317    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548319    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548322    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:18.551974 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548324    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:18.552446 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548327    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:18.552446 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548329    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:18.552446 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548333    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:18.552446 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548336    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:18.552446 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548338    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:18.552446 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548341    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:18.552446 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:18.548344    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:18.552446 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.548349    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:16:18.552446 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.548441    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 18:16:18.552446 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.551492    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 18:16:18.552685 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.552604    2562 server.go:1019] "Starting client certificate rotation"
Sep 16 18:16:18.552729 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.552712    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:16:18.553516 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.553504    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:16:18.555398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.555384    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:16:18.558288 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.558271    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:16:18.568117 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.568095    2562 log.go:25] "Validated CRI v1 runtime API"
Sep 16 18:16:18.571401 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.571384    2562 log.go:25] "Validated CRI v1 image API"
Sep 16 18:16:18.575038 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.575020    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 18:16:18.580956 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.580931    2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 97772522-99ad-4e03-83a6-404e758294bd:/dev/nvme0n1p3 b1260862-4786-4435-b892-dd58773697bd:/dev/nvme0n1p4]
Sep 16 18:16:18.581013 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.580956    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 18:16:18.587452 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.587348    2562 manager.go:216] Machine: {Timestamp:2026-09-16 18:16:18.586873153 +0000 UTC m=+0.351392525 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3198361 MemoryCapacity:32812146688 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2eb1a9c88c5acf021f024a080922e2 SystemUUID:ec2eb1a9-c88c-5acf-021f-024a080922e2 BootID:23bb9c94-79eb-43d7-a2fa-431752ca8ba6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406073344 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:db:20:16:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:db:20:16:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:47:65:11:73:b2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812146688 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 18:16:18.587452 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.587444    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 18:16:18.587567 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.587517    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 18:16:18.589378 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.589356    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 18:16:18.589498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.589381    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-100.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 18:16:18.590057 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.590043    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:16:18.590240 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.590229    2562 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 18:16:18.590280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.590241    2562 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 18:16:18.590280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.590249    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:16:18.592061 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.592051    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:16:18.593282 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.593273    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 18:16:18.596737 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.596726    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:16:18.596842 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.596832    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 18:16:18.596890 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.596876    2562 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 18:16:18.596890 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.596886    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 18:16:18.596944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.596900    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 18:16:18.596944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.596909    2562 kubelet.go:398] "Adding apiserver pod source"
Sep 16 18:16:18.596944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.596916    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 18:16:18.597235 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.597224    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:16:18.597290 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.597243    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:16:18.600419 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.600400    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 18:16:18.603020 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.603006    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 18:16:18.603082 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.603035    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 18:16:18.604968 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.604958    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 18:16:18.605013 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.604973    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 18:16:18.605013 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.604979    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 18:16:18.605013 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.604984    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 18:16:18.605013 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.604991    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 18:16:18.605013 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.604996    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 18:16:18.605013 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.605009    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 18:16:18.605013 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.605014    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 18:16:18.605291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.605021    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 18:16:18.605291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.605026    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 18:16:18.605291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.605034    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 18:16:18.605291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.605039    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 18:16:18.606024 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.606013    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 18:16:18.606024 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.606023    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 18:16:18.607432 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.607414    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-100.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 18:16:18.607432 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.607417    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 18:16:18.607681 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.607658    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-100.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 18:16:18.609731 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.609719    2562 server.go:1268] "Started kubelet"
Sep 16 18:16:18.609916 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.609870    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 18:16:18.609996 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.609934    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 18:16:18.610562 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.610533    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 18:16:18.611066 ip-10-0-133-100 systemd[1]: Started Kubernetes Kubelet.
Sep 16 18:16:18.611653 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.611638    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 18:16:18.613258 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.613242    2562 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 18:16:18.613606 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.613577    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bhvgk"
Sep 16 18:16:18.616229 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.616212    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 18:16:18.618533 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.618515    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 18:16:18.618626 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.618522    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 18:16:18.618686 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.618625    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-100.ec2.internal\" not found"
Sep 16 18:16:18.619314 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619289    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bhvgk"
Sep 16 18:16:18.619434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619315    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 18:16:18.619534 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619447    2562 factory.go:55] Registering systemd factory
Sep 16 18:16:18.619534 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619459    2562 factory.go:223] Registration of the systemd container factory successfully
Sep 16 18:16:18.619534 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619448    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 18:16:18.619534 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619510    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 18:16:18.619703 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619578    2562 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 18:16:18.619703 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619589    2562 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 18:16:18.619703 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619668    2562 factory.go:153] Registering CRI-O factory
Sep 16 18:16:18.619703 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619680    2562 factory.go:223] Registration of the crio container factory successfully
Sep 16 18:16:18.619846 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619729    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 18:16:18.619846 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619749    2562 factory.go:103] Registering Raw factory
Sep 16 18:16:18.619846 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.619762    2562 manager.go:1196] Started watching for new ooms in manager
Sep 16 18:16:18.620492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.620475    2562 manager.go:349] Starting recovery of all containers
Sep 16 18:16:18.623826 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.622776    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 18:16:18.623826 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.623413    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-100.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 18:16:18.625487 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.623041    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-100.ec2.internal.18d5dfdfa6412b9b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-100.ec2.internal,UID:ip-10-0-133-100.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-100.ec2.internal,},FirstTimestamp:2026-09-16 18:16:18.609630107 +0000 UTC m=+0.374149480,LastTimestamp:2026-09-16 18:16:18.609630107 +0000 UTC m=+0.374149480,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-100.ec2.internal,}"
Sep 16 18:16:18.635494 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.635359    2562 manager.go:354] Recovery completed
Sep 16 18:16:18.639375 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.639363    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:18.641138 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.641118    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:18.641211 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.641154    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:18.641211 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.641169    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:18.641703 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.641690    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 18:16:18.641703 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.641702    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 18:16:18.641795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.641719    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:16:18.641795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.641733    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:18.642393 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.642379    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:18.642444 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.642402    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:18.642444 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.642412    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:18.642653 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.642642    2562 policy_none.go:49] "None policy: Start"
Sep 16 18:16:18.642688 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.642656    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 18:16:18.642688 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.642666    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 18:16:18.643584 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.643574    2562 policy_none.go:47] "Start"
Sep 16 18:16:18.675503 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.675490    2562 manager.go:337] "Starting Device Plugin manager"
Sep 16 18:16:18.690484 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.675521    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 18:16:18.690484 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.675530    2562 server.go:87] "Starting device plugin registration server"
Sep 16 18:16:18.690484 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.675778    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 18:16:18.690484 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.675792    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 18:16:18.690484 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.675891    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 18:16:18.690484 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.675980    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 18:16:18.690484 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.675988    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 18:16:18.690484 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.676477    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 18:16:18.690484 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.676522    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-100.ec2.internal\" not found"
Sep 16 18:16:18.752533 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.752510    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 18:16:18.753764 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.753749    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 18:16:18.754264 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.754248    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 18:16:18.754332 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.754277    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 18:16:18.754332 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.754310    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 18:16:18.756907 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.756883    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:18.776099 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.776047    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:18.776741 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.776727    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:18.776798 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.776759    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:18.776798 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.776773    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:18.776798 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.776790    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-100.ec2.internal"
Sep 16 18:16:18.784879 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.784852    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-100.ec2.internal"
Sep 16 18:16:18.784927 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.784885    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-100.ec2.internal\": node \"ip-10-0-133-100.ec2.internal\" not found"
Sep 16 18:16:18.801046 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.801029    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-100.ec2.internal\" not found"
Sep 16 18:16:18.855397 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.855369    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal"]
Sep 16 18:16:18.855477 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.855452    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:18.856978 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.856963    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:18.857061 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.856992    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:18.857061 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.857005    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:18.858101 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.858087    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:18.858178 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.858159    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:18.858231 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.858181    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:18.858730 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.858710    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:18.858730 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.858722    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:18.858844 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.858740    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:18.858844 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.858745    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:18.858844 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.858754    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:18.858844 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.858757    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:18.859759 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.859745    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:18.859792 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.859776    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:18.860422 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.860401    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:18.860478 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.860429    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:18.860478 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.860439    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:18.873414 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.873398    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-100.ec2.internal\" not found" node="ip-10-0-133-100.ec2.internal"
Sep 16 18:16:18.876993 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.876977    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-100.ec2.internal\" not found" node="ip-10-0-133-100.ec2.internal"
Sep 16 18:16:18.901536 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:18.901519    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-100.ec2.internal\" not found"
Sep 16 18:16:18.922072 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.922052    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/daaf67e5c8e04e118d334be2e4278536-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"daaf67e5c8e04e118d334be2e4278536\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:18.922150 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.922080    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/35aa6f7c2ee614a3c7e88ad95244c3eb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-100.ec2.internal\" (UID: \"35aa6f7c2ee614a3c7e88ad95244c3eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:18.922150 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:18.922105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/daaf67e5c8e04e118d334be2e4278536-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"daaf67e5c8e04e118d334be2e4278536\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:19.002466 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.002447    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-100.ec2.internal\" not found"
Sep 16 18:16:19.022759 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.022742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/daaf67e5c8e04e118d334be2e4278536-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"daaf67e5c8e04e118d334be2e4278536\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:19.022805 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.022764    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/daaf67e5c8e04e118d334be2e4278536-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"daaf67e5c8e04e118d334be2e4278536\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:19.022805 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.022780    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/35aa6f7c2ee614a3c7e88ad95244c3eb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-100.ec2.internal\" (UID: \"35aa6f7c2ee614a3c7e88ad95244c3eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:19.022900 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.022842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/35aa6f7c2ee614a3c7e88ad95244c3eb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-100.ec2.internal\" (UID: \"35aa6f7c2ee614a3c7e88ad95244c3eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:19.022900 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.022889    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/daaf67e5c8e04e118d334be2e4278536-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"daaf67e5c8e04e118d334be2e4278536\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:19.022964 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.022918    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/daaf67e5c8e04e118d334be2e4278536-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"daaf67e5c8e04e118d334be2e4278536\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:19.103193 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.103147    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-100.ec2.internal\" not found"
Sep 16 18:16:19.175604 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.175577    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:19.180274 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.180260    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:19.203614 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.203591    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-100.ec2.internal\" not found"
Sep 16 18:16:19.304074 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.304054    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-100.ec2.internal\" not found"
Sep 16 18:16:19.404592 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.404575    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-100.ec2.internal\" not found"
Sep 16 18:16:19.475198 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.475180    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:19.505550 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.505527    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-100.ec2.internal\" not found"
Sep 16 18:16:19.526584 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.526564    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:19.552642 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.552623    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 18:16:19.552743 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.552723    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:16:19.552786 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.552761    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:16:19.552786 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.552774    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:16:19.597952 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.597935    2562 apiserver.go:52] "Watching apiserver"
Sep 16 18:16:19.613673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.613654    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 18:16:19.614100 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.614077    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-p8h5l","openshift-network-operator/iptables-alerter-c4vt7","kube-system/global-pull-secret-syncer-2x4vg","kube-system/konnectivity-agent-4wchr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8","openshift-dns/node-resolver-fxbpm","openshift-image-registry/node-ca-gzdnv","openshift-multus/multus-n9h6f","openshift-ovn-kubernetes/ovnkube-node-c2w7h","openshift-cluster-node-tuning-operator/tuned-9fbvc","openshift-multus/multus-additional-cni-plugins-xnngf","openshift-multus/network-metrics-daemon-mp8m7"]
Sep 16 18:16:19.615417 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.615392    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:19.615548 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.615480    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:19.616246 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.616230    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c4vt7"
Sep 16 18:16:19.617120 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.617097    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:19.617207 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.617173    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:19.618041 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.618027    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:19.618853 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.618833    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:19.618952 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.618873    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 18:16:19.619005 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.618994    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:16:19.619074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.619058    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.619132 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.619115    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 18:16:19.619218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.619203    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d6tnz\""
Sep 16 18:16:19.619597 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.619580    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 18:16:19.620272 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.620252    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fxbpm"
Sep 16 18:16:19.621193 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.621178    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gzdnv"
Sep 16 18:16:19.621831 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.621810    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 18:16:19.621949 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.621923    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 18:16:19.621949 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.621934    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 18:16:19.622157 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.622140    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.622252 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.622174    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 18:16:19.622252 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.622229    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 18:16:19.622352 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.622231    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-twspf\""
Sep 16 18:16:19.622403 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.622390    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dhcfw\""
Sep 16 18:16:19.622791 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.622775    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kmk6v\""
Sep 16 18:16:19.623312 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.623297    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.623413 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.623398    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 18:16:19.624625 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.624427    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 18:16:19.624625 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.624450    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 18:16:19.624625 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.624487    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 18:16:19.624625 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.624497    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 18:16:19.624625 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.624520    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-85n7x\""
Sep 16 18:16:19.624625 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.624608    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.624956 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.624764    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 18:16:19.625012 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.624959    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a820827f-2aeb-4415-b031-2278cde1f521-agent-certs\") pod \"konnectivity-agent-4wchr\" (UID: \"a820827f-2aeb-4415-b031-2278cde1f521\") " pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:19.625012 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.624979    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 18:16:19.625012 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.624977    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-device-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.625148 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625042    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-etc-selinux\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.625148 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625056    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/634753ab-bf55-4898-a91d-7b08037e8ac3-hosts-file\") pod \"node-resolver-fxbpm\" (UID: \"634753ab-bf55-4898-a91d-7b08037e8ac3\") " pod="openshift-dns/node-resolver-fxbpm"
Sep 16 18:16:19.625148 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625071    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8b056af3-46bd-4340-8336-66d3e52a40f7-dbus\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:19.625148 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625106    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv97r\" (UniqueName: \"kubernetes.io/projected/634753ab-bf55-4898-a91d-7b08037e8ac3-kube-api-access-nv97r\") pod \"node-resolver-fxbpm\" (UID: \"634753ab-bf55-4898-a91d-7b08037e8ac3\") " pod="openshift-dns/node-resolver-fxbpm"
Sep 16 18:16:19.625148 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625129    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/892f1cab-d005-4df0-ac49-799b3445cc57-host-slash\") pod \"iptables-alerter-c4vt7\" (UID: \"892f1cab-d005-4df0-ac49-799b3445cc57\") " pod="openshift-network-operator/iptables-alerter-c4vt7"
Sep 16 18:16:19.625384 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625152    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a820827f-2aeb-4415-b031-2278cde1f521-konnectivity-ca\") pod \"konnectivity-agent-4wchr\" (UID: \"a820827f-2aeb-4415-b031-2278cde1f521\") " pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:19.625384 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625186    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-sys-fs\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.625490 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625207    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdpbv\" (UniqueName: \"kubernetes.io/projected/59507886-85d1-4a81-b563-59ef4dd70fb6-kube-api-access-kdpbv\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.625535 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625518    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqjmc\" (UniqueName: \"kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc\") pod \"network-check-target-p8h5l\" (UID: \"9594a6fd-1697-4d79-a706-500845eb00f8\") " pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:19.625581 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/892f1cab-d005-4df0-ac49-799b3445cc57-iptables-alerter-script\") pod \"iptables-alerter-c4vt7\" (UID: \"892f1cab-d005-4df0-ac49-799b3445cc57\") " pod="openshift-network-operator/iptables-alerter-c4vt7"
Sep 16 18:16:19.625626 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625585    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8b056af3-46bd-4340-8336-66d3e52a40f7-kubelet-config\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:19.625626 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625614    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.625716 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625646    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-socket-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.625716 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625669    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-registration-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.625716 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625703    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/634753ab-bf55-4898-a91d-7b08037e8ac3-tmp-dir\") pod \"node-resolver-fxbpm\" (UID: \"634753ab-bf55-4898-a91d-7b08037e8ac3\") " pod="openshift-dns/node-resolver-fxbpm"
Sep 16 18:16:19.625877 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhfsn\" (UniqueName: \"kubernetes.io/projected/892f1cab-d005-4df0-ac49-799b3445cc57-kube-api-access-hhfsn\") pod \"iptables-alerter-c4vt7\" (UID: \"892f1cab-d005-4df0-ac49-799b3445cc57\") " pod="openshift-network-operator/iptables-alerter-c4vt7"
Sep 16 18:16:19.625877 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.625766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:19.626071 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.626052    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.626195 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.626177    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 18:16:19.626816 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.626769    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 18:11:20 +0000 UTC" deadline="2028-03-20 22:46:30.431174847 +0000 UTC"
Sep 16 18:16:19.626905 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.626818    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13228h30m10.804360925s"
Sep 16 18:16:19.627845 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.627790    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 18:16:19.627972 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.627953    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zlfxn\""
Sep 16 18:16:19.628096 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.628082    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.628277    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.628304    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.628324    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.628369    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.628618    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.629040    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.629269    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.629391    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.629400    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fx5tq\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.629425    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qhgwc\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.629496    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-frzft\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.629528    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.629612    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 18:16:19.629797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.629700    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 18:16:19.631633 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.631143    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal"]
Sep 16 18:16:19.631895 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.631879    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:16:19.631953 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.631948    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal"
Sep 16 18:16:19.636176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.636129    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:16:19.641166 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.641150    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal"]
Sep 16 18:16:19.641488 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.641475    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:16:19.661532 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.661491    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nr5sb"
Sep 16 18:16:19.670460 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.670442    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nr5sb"
Sep 16 18:16:19.704391 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:19.704370    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35aa6f7c2ee614a3c7e88ad95244c3eb.slice/crio-50b62338749f9f705ae10a11050e4071f92a8908839a6700c03f045e203406ca WatchSource:0}: Error finding container 50b62338749f9f705ae10a11050e4071f92a8908839a6700c03f045e203406ca: Status 404 returned error can't find the container with id 50b62338749f9f705ae10a11050e4071f92a8908839a6700c03f045e203406ca
Sep 16 18:16:19.704733 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:19.704720    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaaf67e5c8e04e118d334be2e4278536.slice/crio-11dcc17f6b3f91df8da4939c2b5679d64c76f529719ed950785be291aa86eaeb WatchSource:0}: Error finding container 11dcc17f6b3f91df8da4939c2b5679d64c76f529719ed950785be291aa86eaeb: Status 404 returned error can't find the container with id 11dcc17f6b3f91df8da4939c2b5679d64c76f529719ed950785be291aa86eaeb
Sep 16 18:16:19.709221 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.709207    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:16:19.720115 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.720093    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 18:16:19.726494 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-registration-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.726598 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726507    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-cni-binary-copy\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.726598 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726530    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-run-netns\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.726598 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726549    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-sys\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.726598 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726573    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-systemd-units\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.726598 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726587    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-registration-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.726598 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726596    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-run-ovn-kubernetes\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.726898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726621    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87kq5\" (UniqueName: \"kubernetes.io/projected/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-kube-api-access-87kq5\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.726898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726644    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/473ad98e-110c-4d09-a2c6-ac1655312dd8-cni-binary-copy\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.726898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/634753ab-bf55-4898-a91d-7b08037e8ac3-hosts-file\") pod \"node-resolver-fxbpm\" (UID: \"634753ab-bf55-4898-a91d-7b08037e8ac3\") " pod="openshift-dns/node-resolver-fxbpm"
Sep 16 18:16:19.726898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726691    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-var-lib-cni-bin\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.726898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726714    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-hostroot\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.726898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726722    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/634753ab-bf55-4898-a91d-7b08037e8ac3-hosts-file\") pod \"node-resolver-fxbpm\" (UID: \"634753ab-bf55-4898-a91d-7b08037e8ac3\") " pod="openshift-dns/node-resolver-fxbpm"
Sep 16 18:16:19.726898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-sysconfig\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.726898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726771    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.726898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726795    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-ovnkube-script-lib\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.726898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726831    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/473ad98e-110c-4d09-a2c6-ac1655312dd8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.726898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-host\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd9tp\" (UniqueName: \"kubernetes.io/projected/c0001fd5-b780-42e5-a8f8-84f1f01076fe-kube-api-access-cd9tp\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjmc\" (UniqueName: \"kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc\") pod \"network-check-target-p8h5l\" (UID: \"9594a6fd-1697-4d79-a706-500845eb00f8\") " pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.726966    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-modprobe-d\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-lib-modules\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c391df85-454c-4e29-83e5-32a677d165d2-tmp\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727076    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-var-lib-kubelet\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727100    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-slash\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727123    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-var-lib-openvswitch\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727154    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5wwx\" (UniqueName: \"kubernetes.io/projected/473ad98e-110c-4d09-a2c6-ac1655312dd8-kube-api-access-c5wwx\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727209    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8b056af3-46bd-4340-8336-66d3e52a40f7-kubelet-config\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727232    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-socket-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727247    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-run\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727275    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-run-netns\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727298    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-run-openvswitch\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8b056af3-46bd-4340-8336-66d3e52a40f7-kubelet-config\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:19.727318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727321    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-cni-netd\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727349    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/473ad98e-110c-4d09-a2c6-ac1655312dd8-cnibin\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727378    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-socket-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727388    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-etc-selinux\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727404    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-systemd\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727419    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8b056af3-46bd-4340-8336-66d3e52a40f7-dbus\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-etc-selinux\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727457    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-system-cni-dir\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8b056af3-46bd-4340-8336-66d3e52a40f7-dbus\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727490    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-var-lib-kubelet\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727519    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-etc-kubernetes\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727545    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a820827f-2aeb-4415-b031-2278cde1f521-konnectivity-ca\") pod \"konnectivity-agent-4wchr\" (UID: \"a820827f-2aeb-4415-b031-2278cde1f521\") " pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727592    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-run-multus-certs\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727619    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-kubernetes\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727633    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-run-systemd\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a094404a-9871-499f-bbf5-0223671705d4-host\") pod \"node-ca-gzdnv\" (UID: \"a094404a-9871-499f-bbf5-0223671705d4\") " pod="openshift-image-registry/node-ca-gzdnv"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727662    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a094404a-9871-499f-bbf5-0223671705d4-serviceca\") pod \"node-ca-gzdnv\" (UID: \"a094404a-9871-499f-bbf5-0223671705d4\") " pod="openshift-image-registry/node-ca-gzdnv"
Sep 16 18:16:19.728062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c391df85-454c-4e29-83e5-32a677d165d2-etc-tuned\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727713    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t29ch\" (UniqueName: \"kubernetes.io/projected/c391df85-454c-4e29-83e5-32a677d165d2-kube-api-access-t29ch\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727744    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-ovn-node-metrics-cert\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-device-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727814    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-device-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727812    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbngm\" (UniqueName: \"kubernetes.io/projected/a094404a-9871-499f-bbf5-0223671705d4-kube-api-access-qbngm\") pod \"node-ca-gzdnv\" (UID: \"a094404a-9871-499f-bbf5-0223671705d4\") " pod="openshift-image-registry/node-ca-gzdnv"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-run-k8s-cni-cncf-io\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727897    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-sysctl-conf\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727915    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/473ad98e-110c-4d09-a2c6-ac1655312dd8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727935    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/892f1cab-d005-4df0-ac49-799b3445cc57-host-slash\") pod \"iptables-alerter-c4vt7\" (UID: \"892f1cab-d005-4df0-ac49-799b3445cc57\") " pod="openshift-network-operator/iptables-alerter-c4vt7"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727955    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-kubelet\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727978    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-run-ovn\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.727985    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/892f1cab-d005-4df0-ac49-799b3445cc57-host-slash\") pod \"iptables-alerter-c4vt7\" (UID: \"892f1cab-d005-4df0-ac49-799b3445cc57\") " pod="openshift-network-operator/iptables-alerter-c4vt7"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728002    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-sys-fs\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728031    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/892f1cab-d005-4df0-ac49-799b3445cc57-iptables-alerter-script\") pod \"iptables-alerter-c4vt7\" (UID: \"892f1cab-d005-4df0-ac49-799b3445cc57\") " pod="openshift-network-operator/iptables-alerter-c4vt7"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728047    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-sys-fs\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728052    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-conf-dir\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.728829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728069    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a820827f-2aeb-4415-b031-2278cde1f521-konnectivity-ca\") pod \"konnectivity-agent-4wchr\" (UID: \"a820827f-2aeb-4415-b031-2278cde1f521\") " pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728071    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jjtw\" (UniqueName: \"kubernetes.io/projected/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-kube-api-access-2jjtw\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728146    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-sysctl-d\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728170    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-etc-openvswitch\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728193    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-node-log\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-log-socket\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728235    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-cnibin\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728325    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/634753ab-bf55-4898-a91d-7b08037e8ac3-tmp-dir\") pod \"node-resolver-fxbpm\" (UID: \"634753ab-bf55-4898-a91d-7b08037e8ac3\") " pod="openshift-dns/node-resolver-fxbpm"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhfsn\" (UniqueName: \"kubernetes.io/projected/892f1cab-d005-4df0-ac49-799b3445cc57-kube-api-access-hhfsn\") pod \"iptables-alerter-c4vt7\" (UID: \"892f1cab-d005-4df0-ac49-799b3445cc57\") " pod="openshift-network-operator/iptables-alerter-c4vt7"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-daemon-config\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728377    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/59507886-85d1-4a81-b563-59ef4dd70fb6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728396    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-ovnkube-config\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728456    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/473ad98e-110c-4d09-a2c6-ac1655312dd8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728462    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/892f1cab-d005-4df0-ac49-799b3445cc57-iptables-alerter-script\") pod \"iptables-alerter-c4vt7\" (UID: \"892f1cab-d005-4df0-ac49-799b3445cc57\") " pod="openshift-network-operator/iptables-alerter-c4vt7"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728515    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a820827f-2aeb-4415-b031-2278cde1f521-agent-certs\") pod \"konnectivity-agent-4wchr\" (UID: \"a820827f-2aeb-4415-b031-2278cde1f521\") " pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:19.729438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728539    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-cni-dir\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728563    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-socket-dir-parent\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728588    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.728600    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728610    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-env-overrides\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728637    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv97r\" (UniqueName: \"kubernetes.io/projected/634753ab-bf55-4898-a91d-7b08037e8ac3-kube-api-access-nv97r\") pod \"node-resolver-fxbpm\" (UID: \"634753ab-bf55-4898-a91d-7b08037e8ac3\") " pod="openshift-dns/node-resolver-fxbpm"
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.728676    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret podName:8b056af3-46bd-4340-8336-66d3e52a40f7 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:20.228654906 +0000 UTC m=+1.993174288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret") pod "global-pull-secret-syncer-2x4vg" (UID: "8b056af3-46bd-4340-8336-66d3e52a40f7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-var-lib-cni-multus\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728779    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-cni-bin\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728808    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/473ad98e-110c-4d09-a2c6-ac1655312dd8-system-cni-dir\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.728834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdpbv\" (UniqueName: \"kubernetes.io/projected/59507886-85d1-4a81-b563-59ef4dd70fb6-kube-api-access-kdpbv\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.729043    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/634753ab-bf55-4898-a91d-7b08037e8ac3-tmp-dir\") pod \"node-resolver-fxbpm\" (UID: \"634753ab-bf55-4898-a91d-7b08037e8ac3\") " pod="openshift-dns/node-resolver-fxbpm"
Sep 16 18:16:19.729908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.729480    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 18:16:19.732627 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.732605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a820827f-2aeb-4415-b031-2278cde1f521-agent-certs\") pod \"konnectivity-agent-4wchr\" (UID: \"a820827f-2aeb-4415-b031-2278cde1f521\") " pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:19.733577 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.733559    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:19.733577 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.733576    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:19.733713 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.733585    2562 projected.go:196] Error preparing data for projected volume kube-api-access-kqjmc for pod openshift-network-diagnostics/network-check-target-p8h5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:19.733713 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.733635    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc podName:9594a6fd-1697-4d79-a706-500845eb00f8 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:20.233622665 +0000 UTC m=+1.998142025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kqjmc" (UniqueName: "kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc") pod "network-check-target-p8h5l" (UID: "9594a6fd-1697-4d79-a706-500845eb00f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:19.741195 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.741175    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv97r\" (UniqueName: \"kubernetes.io/projected/634753ab-bf55-4898-a91d-7b08037e8ac3-kube-api-access-nv97r\") pod \"node-resolver-fxbpm\" (UID: \"634753ab-bf55-4898-a91d-7b08037e8ac3\") " pod="openshift-dns/node-resolver-fxbpm"
Sep 16 18:16:19.741288 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.741233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhfsn\" (UniqueName: \"kubernetes.io/projected/892f1cab-d005-4df0-ac49-799b3445cc57-kube-api-access-hhfsn\") pod \"iptables-alerter-c4vt7\" (UID: \"892f1cab-d005-4df0-ac49-799b3445cc57\") " pod="openshift-network-operator/iptables-alerter-c4vt7"
Sep 16 18:16:19.741436 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.741421    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdpbv\" (UniqueName: \"kubernetes.io/projected/59507886-85d1-4a81-b563-59ef4dd70fb6-kube-api-access-kdpbv\") pod \"aws-ebs-csi-driver-node-wwbp8\" (UID: \"59507886-85d1-4a81-b563-59ef4dd70fb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.757060 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.757013    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" event={"ID":"35aa6f7c2ee614a3c7e88ad95244c3eb","Type":"ContainerStarted","Data":"50b62338749f9f705ae10a11050e4071f92a8908839a6700c03f045e203406ca"}
Sep 16 18:16:19.757987 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.757961    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" event={"ID":"daaf67e5c8e04e118d334be2e4278536","Type":"ContainerStarted","Data":"11dcc17f6b3f91df8da4939c2b5679d64c76f529719ed950785be291aa86eaeb"}
Sep 16 18:16:19.829063 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-modprobe-d\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.829063 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829063    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-lib-modules\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.829187 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829077    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c391df85-454c-4e29-83e5-32a677d165d2-tmp\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.829187 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829092    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-var-lib-kubelet\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.829187 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829105    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-slash\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.829187 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-var-lib-openvswitch\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.829187 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829150    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5wwx\" (UniqueName: \"kubernetes.io/projected/473ad98e-110c-4d09-a2c6-ac1655312dd8-kube-api-access-c5wwx\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.829187 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-run\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.829187 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829178    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-var-lib-kubelet\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829192    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-run-netns\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829178    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-lib-modules\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829198    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-slash\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829214    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-run-openvswitch\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829213    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-modprobe-d\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-var-lib-openvswitch\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829245    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-run\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829260    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-run-netns\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-cni-netd\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/473ad98e-110c-4d09-a2c6-ac1655312dd8-cnibin\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829259    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-run-openvswitch\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829325    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-systemd\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-system-cni-dir\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829354    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/473ad98e-110c-4d09-a2c6-ac1655312dd8-cnibin\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829372    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-var-lib-kubelet\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-cni-netd\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-systemd\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.829498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-etc-kubernetes\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829408    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-var-lib-kubelet\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829420    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-system-cni-dir\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829421    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-run-multus-certs\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829447    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-etc-kubernetes\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829449    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-kubernetes\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829459    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-run-multus-certs\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-run-systemd\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829477    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-kubernetes\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a094404a-9871-499f-bbf5-0223671705d4-host\") pod \"node-ca-gzdnv\" (UID: \"a094404a-9871-499f-bbf5-0223671705d4\") " pod="openshift-image-registry/node-ca-gzdnv"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829503    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-run-systemd\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829544    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a094404a-9871-499f-bbf5-0223671705d4-serviceca\") pod \"node-ca-gzdnv\" (UID: \"a094404a-9871-499f-bbf5-0223671705d4\") " pod="openshift-image-registry/node-ca-gzdnv"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829558    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c391df85-454c-4e29-83e5-32a677d165d2-etc-tuned\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829573    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t29ch\" (UniqueName: \"kubernetes.io/projected/c391df85-454c-4e29-83e5-32a677d165d2-kube-api-access-t29ch\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829580    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a094404a-9871-499f-bbf5-0223671705d4-host\") pod \"node-ca-gzdnv\" (UID: \"a094404a-9871-499f-bbf5-0223671705d4\") " pod="openshift-image-registry/node-ca-gzdnv"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829594    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-ovn-node-metrics-cert\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbngm\" (UniqueName: \"kubernetes.io/projected/a094404a-9871-499f-bbf5-0223671705d4-kube-api-access-qbngm\") pod \"node-ca-gzdnv\" (UID: \"a094404a-9871-499f-bbf5-0223671705d4\") " pod="openshift-image-registry/node-ca-gzdnv"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829638    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-run-k8s-cni-cncf-io\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.830521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829659    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-sysctl-conf\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/473ad98e-110c-4d09-a2c6-ac1655312dd8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-kubelet\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829733    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-run-ovn\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829895    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-kubelet\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829939    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-run-ovn\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829938    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a094404a-9871-499f-bbf5-0223671705d4-serviceca\") pod \"node-ca-gzdnv\" (UID: \"a094404a-9871-499f-bbf5-0223671705d4\") " pod="openshift-image-registry/node-ca-gzdnv"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.829976    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-run-k8s-cni-cncf-io\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-sysctl-conf\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830090    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-conf-dir\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jjtw\" (UniqueName: \"kubernetes.io/projected/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-kube-api-access-2jjtw\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-sysctl-d\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-etc-openvswitch\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-conf-dir\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830180    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-node-log\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830202    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-log-socket\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-cnibin\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-daemon-config\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.831372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830261    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-sysctl-d\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-ovnkube-config\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830305    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/473ad98e-110c-4d09-a2c6-ac1655312dd8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830340    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-cni-dir\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830376    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-socket-dir-parent\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830403    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830425    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-env-overrides\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-var-lib-cni-multus\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830477    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/473ad98e-110c-4d09-a2c6-ac1655312dd8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830489    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-cni-bin\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830496    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-node-log\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830511    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/473ad98e-110c-4d09-a2c6-ac1655312dd8-system-cni-dir\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830532    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-cnibin\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830537    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-cni-binary-copy\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830559    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-run-netns\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830569    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-log-socket\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830580    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-sys\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.832126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830602    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-systemd-units\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830623    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-run-ovn-kubernetes\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830647    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87kq5\" (UniqueName: \"kubernetes.io/projected/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-kube-api-access-87kq5\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830683    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/473ad98e-110c-4d09-a2c6-ac1655312dd8-cni-binary-copy\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-var-lib-cni-bin\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-hostroot\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830752    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-sysconfig\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830776    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-ovnkube-script-lib\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830837    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/473ad98e-110c-4d09-a2c6-ac1655312dd8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-host\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830944    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-host\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-env-overrides\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-etc-openvswitch\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830983    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd9tp\" (UniqueName: \"kubernetes.io/projected/c0001fd5-b780-42e5-a8f8-84f1f01076fe-kube-api-access-cd9tp\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831023    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-socket-dir-parent\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.831042    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831052    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-daemon-config\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.832576 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831062    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-cni-bin\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:19.831084    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs podName:c0001fd5-b780-42e5-a8f8-84f1f01076fe nodeName:}" failed. No retries permitted until 2026-09-16 18:16:20.331069629 +0000 UTC m=+2.095589007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs") pod "network-metrics-daemon-mp8m7" (UID: "c0001fd5-b780-42e5-a8f8-84f1f01076fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.830953    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-multus-cni-dir\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831110    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-run-netns\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/473ad98e-110c-4d09-a2c6-ac1655312dd8-system-cni-dir\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-systemd-units\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831196    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-hostroot\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831224    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-etc-sysconfig\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831237    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-run-ovn-kubernetes\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831519    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-cni-binary-copy\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831577    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c391df85-454c-4e29-83e5-32a677d165d2-sys\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831594    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/473ad98e-110c-4d09-a2c6-ac1655312dd8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-var-lib-cni-multus\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831712    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-ovnkube-config\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831730    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/473ad98e-110c-4d09-a2c6-ac1655312dd8-cni-binary-copy\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831742    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-ovnkube-script-lib\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.833055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831840    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c391df85-454c-4e29-83e5-32a677d165d2-etc-tuned\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.833492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831853    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-host-var-lib-cni-bin\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.833492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c391df85-454c-4e29-83e5-32a677d165d2-tmp\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.833492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.831945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-ovn-node-metrics-cert\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.833492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.832008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/473ad98e-110c-4d09-a2c6-ac1655312dd8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.841837 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.841815    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jjtw\" (UniqueName: \"kubernetes.io/projected/a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4-kube-api-access-2jjtw\") pod \"multus-n9h6f\" (UID: \"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4\") " pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:19.842092 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.842074    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbngm\" (UniqueName: \"kubernetes.io/projected/a094404a-9871-499f-bbf5-0223671705d4-kube-api-access-qbngm\") pod \"node-ca-gzdnv\" (UID: \"a094404a-9871-499f-bbf5-0223671705d4\") " pod="openshift-image-registry/node-ca-gzdnv"
Sep 16 18:16:19.842180 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.842159    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5wwx\" (UniqueName: \"kubernetes.io/projected/473ad98e-110c-4d09-a2c6-ac1655312dd8-kube-api-access-c5wwx\") pod \"multus-additional-cni-plugins-xnngf\" (UID: \"473ad98e-110c-4d09-a2c6-ac1655312dd8\") " pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:19.842727 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.842708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t29ch\" (UniqueName: \"kubernetes.io/projected/c391df85-454c-4e29-83e5-32a677d165d2-kube-api-access-t29ch\") pod \"tuned-9fbvc\" (UID: \"c391df85-454c-4e29-83e5-32a677d165d2\") " pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:19.842890 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.842877    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd9tp\" (UniqueName: \"kubernetes.io/projected/c0001fd5-b780-42e5-a8f8-84f1f01076fe-kube-api-access-cd9tp\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:19.843733 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.843717    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87kq5\" (UniqueName: \"kubernetes.io/projected/6ae2c74c-dbdb-46f2-8c29-e626d3cc052a-kube-api-access-87kq5\") pod \"ovnkube-node-c2w7h\" (UID: \"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:19.943872 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.943817    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c4vt7"
Sep 16 18:16:19.951721 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:19.951702    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod892f1cab_d005_4df0_ac49_799b3445cc57.slice/crio-05ac5d7ef743afab2bcc77e63be54d10a11c35d45ab2566ea96c12c4de7bdbb7 WatchSource:0}: Error finding container 05ac5d7ef743afab2bcc77e63be54d10a11c35d45ab2566ea96c12c4de7bdbb7: Status 404 returned error can't find the container with id 05ac5d7ef743afab2bcc77e63be54d10a11c35d45ab2566ea96c12c4de7bdbb7
Sep 16 18:16:19.960038 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.960024    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:19.965980 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:19.965960    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda820827f_2aeb_4415_b031_2278cde1f521.slice/crio-0eeb4a0e4f25b28c0a161f454e68e97a50974b1053195c7fa12223db6233d4dc WatchSource:0}: Error finding container 0eeb4a0e4f25b28c0a161f454e68e97a50974b1053195c7fa12223db6233d4dc: Status 404 returned error can't find the container with id 0eeb4a0e4f25b28c0a161f454e68e97a50974b1053195c7fa12223db6233d4dc
Sep 16 18:16:19.977078 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.977061    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8"
Sep 16 18:16:19.982946 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:19.982925    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59507886_85d1_4a81_b563_59ef4dd70fb6.slice/crio-7f3f69cf8a97c49119f65251bb07bc909165d78a360a19084556b42056765563 WatchSource:0}: Error finding container 7f3f69cf8a97c49119f65251bb07bc909165d78a360a19084556b42056765563: Status 404 returned error can't find the container with id 7f3f69cf8a97c49119f65251bb07bc909165d78a360a19084556b42056765563
Sep 16 18:16:19.987800 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:19.987786    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fxbpm"
Sep 16 18:16:19.993875 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:19.993732    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634753ab_bf55_4898_a91d_7b08037e8ac3.slice/crio-c69746f29b8c89559d98970dc0f10299b8fe0fac3a6acba4a509a75c11d4f913 WatchSource:0}: Error finding container c69746f29b8c89559d98970dc0f10299b8fe0fac3a6acba4a509a75c11d4f913: Status 404 returned error can't find the container with id c69746f29b8c89559d98970dc0f10299b8fe0fac3a6acba4a509a75c11d4f913
Sep 16 18:16:20.001401 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:20.001385    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:20.011259 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:20.011242    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gzdnv"
Sep 16 18:16:20.017208 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:20.017196    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n9h6f"
Sep 16 18:16:20.017613 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:20.017592    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda094404a_9871_499f_bbf5_0223671705d4.slice/crio-a83fe12836b138a7582eee479f8e08a1e318dab22e7a6bbe597fe8c4751a89e9 WatchSource:0}: Error finding container a83fe12836b138a7582eee479f8e08a1e318dab22e7a6bbe597fe8c4751a89e9: Status 404 returned error can't find the container with id a83fe12836b138a7582eee479f8e08a1e318dab22e7a6bbe597fe8c4751a89e9
Sep 16 18:16:20.022439 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:20.022419    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:20.024096 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:20.024006    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81ed7be_fe2e_49d3_9dd8_0aa0511aaaa4.slice/crio-11709ca448a8b334b862e62094d1899f6f8ec80e1197cfe99c6d929cdc0e46f9 WatchSource:0}: Error finding container 11709ca448a8b334b862e62094d1899f6f8ec80e1197cfe99c6d929cdc0e46f9: Status 404 returned error can't find the container with id 11709ca448a8b334b862e62094d1899f6f8ec80e1197cfe99c6d929cdc0e46f9
Sep 16 18:16:20.028816 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:20.028797    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9fbvc"
Sep 16 18:16:20.030321 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:20.030302    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae2c74c_dbdb_46f2_8c29_e626d3cc052a.slice/crio-712f41d3238246d94fadea3ccd9b6fa922fc197f5d708e33113641dd36a5df67 WatchSource:0}: Error finding container 712f41d3238246d94fadea3ccd9b6fa922fc197f5d708e33113641dd36a5df67: Status 404 returned error can't find the container with id 712f41d3238246d94fadea3ccd9b6fa922fc197f5d708e33113641dd36a5df67
Sep 16 18:16:20.032949 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:20.032928    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xnngf"
Sep 16 18:16:20.037774 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:20.037755    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc391df85_454c_4e29_83e5_32a677d165d2.slice/crio-3fc1eef7f381d8b671023e80a09777a148cb6390afc8fb003db273ce98cd1e54 WatchSource:0}: Error finding container 3fc1eef7f381d8b671023e80a09777a148cb6390afc8fb003db273ce98cd1e54: Status 404 returned error can't find the container with id 3fc1eef7f381d8b671023e80a09777a148cb6390afc8fb003db273ce98cd1e54
Sep 16 18:16:20.040879 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:20.040847    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod473ad98e_110c_4d09_a2c6_ac1655312dd8.slice/crio-262059e8dbc34192738a0377a4d226541f3b2b45aa95dedb1b2c10013d7d9878 WatchSource:0}: Error finding container 262059e8dbc34192738a0377a4d226541f3b2b45aa95dedb1b2c10013d7d9878: Status 404 returned error can't find the container with id 262059e8dbc34192738a0377a4d226541f3b2b45aa95dedb1b2c10013d7d9878
Sep 16 18:16:20.234164 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:20.234096    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:20.234308 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:20.234187    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjmc\" (UniqueName: \"kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc\") pod \"network-check-target-p8h5l\" (UID: \"9594a6fd-1697-4d79-a706-500845eb00f8\") " pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:20.235843 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:20.235817    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:20.235953 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:20.235912    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret podName:8b056af3-46bd-4340-8336-66d3e52a40f7 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:21.235893378 +0000 UTC m=+3.000412751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret") pod "global-pull-secret-syncer-2x4vg" (UID: "8b056af3-46bd-4340-8336-66d3e52a40f7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:20.237884 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:20.236166    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:20.237884 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:20.236193    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:20.237884 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:20.236216    2562 projected.go:196] Error preparing data for projected volume kube-api-access-kqjmc for pod openshift-network-diagnostics/network-check-target-p8h5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:20.237884 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:20.236276    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc podName:9594a6fd-1697-4d79-a706-500845eb00f8 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:21.236251884 +0000 UTC m=+3.000771256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqjmc" (UniqueName: "kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc") pod "network-check-target-p8h5l" (UID: "9594a6fd-1697-4d79-a706-500845eb00f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:20.335435 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:20.335397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:20.335566 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:20.335523    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:20.335631 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:20.335592    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs podName:c0001fd5-b780-42e5-a8f8-84f1f01076fe nodeName:}" failed. No retries permitted until 2026-09-16 18:16:21.335575058 +0000 UTC m=+3.100094424 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs") pod "network-metrics-daemon-mp8m7" (UID: "c0001fd5-b780-42e5-a8f8-84f1f01076fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:21.920543 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:21.917196    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:22.196664 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.196607    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:11:21 +0000 UTC" deadline="2028-03-20 09:25:54.888178035 +0000 UTC"
Sep 16 18:16:22.196664 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.196661    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13215h9m32.691520814s"
Sep 16 18:16:22.309667 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.309554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnngf" event={"ID":"473ad98e-110c-4d09-a2c6-ac1655312dd8","Type":"ContainerStarted","Data":"262059e8dbc34192738a0377a4d226541f3b2b45aa95dedb1b2c10013d7d9878"}
Sep 16 18:16:22.316775 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.316745    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9fbvc" event={"ID":"c391df85-454c-4e29-83e5-32a677d165d2","Type":"ContainerStarted","Data":"3fc1eef7f381d8b671023e80a09777a148cb6390afc8fb003db273ce98cd1e54"}
Sep 16 18:16:22.327011 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.326870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n9h6f" event={"ID":"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4","Type":"ContainerStarted","Data":"11709ca448a8b334b862e62094d1899f6f8ec80e1197cfe99c6d929cdc0e46f9"}
Sep 16 18:16:22.332618 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.332567    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fxbpm" event={"ID":"634753ab-bf55-4898-a91d-7b08037e8ac3","Type":"ContainerStarted","Data":"c69746f29b8c89559d98970dc0f10299b8fe0fac3a6acba4a509a75c11d4f913"}
Sep 16 18:16:22.347391 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.347358    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c4vt7" event={"ID":"892f1cab-d005-4df0-ac49-799b3445cc57","Type":"ContainerStarted","Data":"05ac5d7ef743afab2bcc77e63be54d10a11c35d45ab2566ea96c12c4de7bdbb7"}
Sep 16 18:16:22.364028 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.363999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h" event={"ID":"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a","Type":"ContainerStarted","Data":"712f41d3238246d94fadea3ccd9b6fa922fc197f5d708e33113641dd36a5df67"}
Sep 16 18:16:22.377385 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.377314    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gzdnv" event={"ID":"a094404a-9871-499f-bbf5-0223671705d4","Type":"ContainerStarted","Data":"a83fe12836b138a7582eee479f8e08a1e318dab22e7a6bbe597fe8c4751a89e9"}
Sep 16 18:16:22.387927 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.387857    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8" event={"ID":"59507886-85d1-4a81-b563-59ef4dd70fb6","Type":"ContainerStarted","Data":"7f3f69cf8a97c49119f65251bb07bc909165d78a360a19084556b42056765563"}
Sep 16 18:16:22.411396 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.411367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4wchr" event={"ID":"a820827f-2aeb-4415-b031-2278cde1f521","Type":"ContainerStarted","Data":"0eeb4a0e4f25b28c0a161f454e68e97a50974b1053195c7fa12223db6233d4dc"}
Sep 16 18:16:22.766787 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.766752    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:22.766979 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.766827    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjmc\" (UniqueName: \"kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc\") pod \"network-check-target-p8h5l\" (UID: \"9594a6fd-1697-4d79-a706-500845eb00f8\") " pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:22.767060 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:22.767002    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:22.767060 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:22.767029    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:22.767060 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:22.767043    2562 projected.go:196] Error preparing data for projected volume kube-api-access-kqjmc for pod openshift-network-diagnostics/network-check-target-p8h5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:22.767214 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:22.767098    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc podName:9594a6fd-1697-4d79-a706-500845eb00f8 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:24.767080982 +0000 UTC m=+5.006869717 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqjmc" (UniqueName: "kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc") pod "network-check-target-p8h5l" (UID: "9594a6fd-1697-4d79-a706-500845eb00f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:22.767555 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:22.767518    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:22.767670 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:22.767572    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret podName:8b056af3-46bd-4340-8336-66d3e52a40f7 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:24.767556983 +0000 UTC m=+5.007345719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret") pod "global-pull-secret-syncer-2x4vg" (UID: "8b056af3-46bd-4340-8336-66d3e52a40f7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:22.868338 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.867832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:22.868338 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:22.867982    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:22.868338 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:22.868038    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs podName:c0001fd5-b780-42e5-a8f8-84f1f01076fe nodeName:}" failed. No retries permitted until 2026-09-16 18:16:24.868021882 +0000 UTC m=+5.107810616 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs") pod "network-metrics-daemon-mp8m7" (UID: "c0001fd5-b780-42e5-a8f8-84f1f01076fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:22.984313 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:22.984281    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:23.197774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:23.197734    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:11:21 +0000 UTC" deadline="2028-06-30 04:14:00.147295604 +0000 UTC"
Sep 16 18:16:23.197774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:23.197773    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15657h57m36.949526893s"
Sep 16 18:16:23.279443 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:23.279413    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:23.279715 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:23.279547    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:23.279715 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:23.279644    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:23.279832 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:23.279717    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:23.279832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:23.279777    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:23.279931 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:23.279852    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:23.366054 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:23.366026    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:24.784188 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:24.784148    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:24.784635 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:24.784228    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjmc\" (UniqueName: \"kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc\") pod \"network-check-target-p8h5l\" (UID: \"9594a6fd-1697-4d79-a706-500845eb00f8\") " pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:24.784635 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:24.784387    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:24.784635 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:24.784408    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:24.784635 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:24.784422    2562 projected.go:196] Error preparing data for projected volume kube-api-access-kqjmc for pod openshift-network-diagnostics/network-check-target-p8h5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:24.784635 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:24.784479    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc podName:9594a6fd-1697-4d79-a706-500845eb00f8 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:28.784461167 +0000 UTC m=+9.024249913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqjmc" (UniqueName: "kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc") pod "network-check-target-p8h5l" (UID: "9594a6fd-1697-4d79-a706-500845eb00f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:24.784944 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:24.784925    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:24.785014 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:24.784978    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret podName:8b056af3-46bd-4340-8336-66d3e52a40f7 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:28.784963579 +0000 UTC m=+9.024752323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret") pod "global-pull-secret-syncer-2x4vg" (UID: "8b056af3-46bd-4340-8336-66d3e52a40f7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:24.885145 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:24.885107    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:24.885312 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:24.885285    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:24.885379 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:24.885344    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs podName:c0001fd5-b780-42e5-a8f8-84f1f01076fe nodeName:}" failed. No retries permitted until 2026-09-16 18:16:28.885326477 +0000 UTC m=+9.125115209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs") pod "network-metrics-daemon-mp8m7" (UID: "c0001fd5-b780-42e5-a8f8-84f1f01076fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:25.279230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:25.279197    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:25.279230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:25.279230    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:25.279472 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:25.279207    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:25.279472 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:25.279322    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:25.279472 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:25.279418    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:25.279634 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:25.279517    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:27.279789 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:27.279755    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:27.280209 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:27.279912    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:27.280296 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:27.280277    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:27.280399 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:27.280380    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:27.280455 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:27.280437    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:27.280544 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:27.280505    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:28.821209 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:28.821173    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjmc\" (UniqueName: \"kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc\") pod \"network-check-target-p8h5l\" (UID: \"9594a6fd-1697-4d79-a706-500845eb00f8\") " pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:28.821792 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:28.821245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:28.821792 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:28.821403    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:28.821792 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:28.821461    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret podName:8b056af3-46bd-4340-8336-66d3e52a40f7 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:36.821444519 +0000 UTC m=+17.061233254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret") pod "global-pull-secret-syncer-2x4vg" (UID: "8b056af3-46bd-4340-8336-66d3e52a40f7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:28.821792 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:28.821759    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:28.821792 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:28.821781    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:28.821792 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:28.821794    2562 projected.go:196] Error preparing data for projected volume kube-api-access-kqjmc for pod openshift-network-diagnostics/network-check-target-p8h5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:28.822097 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:28.821854    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc podName:9594a6fd-1697-4d79-a706-500845eb00f8 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:36.821829523 +0000 UTC m=+17.061618275 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqjmc" (UniqueName: "kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc") pod "network-check-target-p8h5l" (UID: "9594a6fd-1697-4d79-a706-500845eb00f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:28.922460 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:28.922426    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:28.922643 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:28.922547    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:28.922643 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:28.922623    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs podName:c0001fd5-b780-42e5-a8f8-84f1f01076fe nodeName:}" failed. No retries permitted until 2026-09-16 18:16:36.922605335 +0000 UTC m=+17.162394067 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs") pod "network-metrics-daemon-mp8m7" (UID: "c0001fd5-b780-42e5-a8f8-84f1f01076fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:29.279780 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:29.279746    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:29.279780 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:29.279774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:29.279980 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:29.279746    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:29.279980 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:29.279895    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:29.279980 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:29.279940    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:29.280076 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:29.280008    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:31.280163 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:31.280129    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:31.280677 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:31.280127    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:31.280677 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:31.280259    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:31.280677 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:31.280143    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:31.280677 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:31.280324    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:31.280677 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:31.280415    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:33.279219 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:33.279186    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:33.279701 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:33.279229    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:33.279701 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:33.279310    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:33.279701 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:33.279311    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:33.279701 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:33.279445    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:33.279701 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:33.279529    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:35.279995 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:35.279965    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:35.280384 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:35.280010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:35.280384 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:35.280090    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:35.280384 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:35.280118    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:35.280384 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:35.280152    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:35.280384 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:35.280233    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:36.884487 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:36.884448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjmc\" (UniqueName: \"kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc\") pod \"network-check-target-p8h5l\" (UID: \"9594a6fd-1697-4d79-a706-500845eb00f8\") " pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:36.884994 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:36.884520    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:36.884994 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:36.884648    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:36.884994 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:36.884649    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:36.884994 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:36.884678    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:36.884994 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:36.884692    2562 projected.go:196] Error preparing data for projected volume kube-api-access-kqjmc for pod openshift-network-diagnostics/network-check-target-p8h5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:36.884994 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:36.884713    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret podName:8b056af3-46bd-4340-8336-66d3e52a40f7 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:52.884694011 +0000 UTC m=+33.124482742 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret") pod "global-pull-secret-syncer-2x4vg" (UID: "8b056af3-46bd-4340-8336-66d3e52a40f7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:36.884994 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:36.884744    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc podName:9594a6fd-1697-4d79-a706-500845eb00f8 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:52.884729176 +0000 UTC m=+33.124517920 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqjmc" (UniqueName: "kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc") pod "network-check-target-p8h5l" (UID: "9594a6fd-1697-4d79-a706-500845eb00f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:36.984849 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:36.984817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:36.985012 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:36.984987    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:36.985125 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:36.985059    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs podName:c0001fd5-b780-42e5-a8f8-84f1f01076fe nodeName:}" failed. No retries permitted until 2026-09-16 18:16:52.985041043 +0000 UTC m=+33.224829777 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs") pod "network-metrics-daemon-mp8m7" (UID: "c0001fd5-b780-42e5-a8f8-84f1f01076fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:37.279423 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:37.279393    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:37.279607 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:37.279394    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:37.279607 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:37.279506    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:37.279607 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:37.279394    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:37.279607 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:37.279582    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:37.279781 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:37.279662    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:39.279982 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:39.279489    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:39.279982 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:39.279499    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:39.279982 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:39.279646    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:39.279982 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:39.279684    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:39.279982 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:39.279754    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:39.279982 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:39.279830    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:40.459749 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.459363    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h" event={"ID":"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a","Type":"ContainerStarted","Data":"345d16d53f3e79099aaea25c365f8401c8ccccb8fdd85f742de1a4d856b8772f"}
Sep 16 18:16:40.460372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.459798    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h" event={"ID":"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a","Type":"ContainerStarted","Data":"16459c9989d816b363b52f552c6069d1f75d972acab29c4be8a66ca2ed54cdbf"}
Sep 16 18:16:40.460372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.459851    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h" event={"ID":"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a","Type":"ContainerStarted","Data":"9fbc2d9068694ffcfd850797bdafbdbdedbdbc822b2a3d23b765be310576c540"}
Sep 16 18:16:40.460372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.459902    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h" event={"ID":"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a","Type":"ContainerStarted","Data":"391aa73eb2a7e67d6ad92753469e5e5c2593fe4f904c82d41547a8697f20351b"}
Sep 16 18:16:40.460711 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.460612    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gzdnv" event={"ID":"a094404a-9871-499f-bbf5-0223671705d4","Type":"ContainerStarted","Data":"ee560aaf74cdaf83417fc5819b097716d6a1e39d5a041b0127a8fc39623916ce"}
Sep 16 18:16:40.462260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.462238    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8" event={"ID":"59507886-85d1-4a81-b563-59ef4dd70fb6","Type":"ContainerStarted","Data":"e93ec5155c85b22854e0cb1a5ee67d8eb5a16fc2e99e2492d1323a8ee813a2ef"}
Sep 16 18:16:40.463692 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.463660    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4wchr" event={"ID":"a820827f-2aeb-4415-b031-2278cde1f521","Type":"ContainerStarted","Data":"2afef2886f17e17a4df2d9756ba82ea1d53c924825b9620f09cc18e011b60574"}
Sep 16 18:16:40.467999 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.467979    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" event={"ID":"35aa6f7c2ee614a3c7e88ad95244c3eb","Type":"ContainerStarted","Data":"b822f9c0b3b5d78e6cbe95536e6e4ab24a7990d95377a8fdbd81d8e7c4dfd0f4"}
Sep 16 18:16:40.469467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.469436    2562 generic.go:356] "Generic (PLEG): container finished" podID="daaf67e5c8e04e118d334be2e4278536" containerID="a9ba59b707f876c1f088f43a6e6a956772a2bb0dab57a726452bcf92f5157d0f" exitCode=0
Sep 16 18:16:40.469541 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.469468    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" event={"ID":"daaf67e5c8e04e118d334be2e4278536","Type":"ContainerDied","Data":"a9ba59b707f876c1f088f43a6e6a956772a2bb0dab57a726452bcf92f5157d0f"}
Sep 16 18:16:40.471029 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.471010    2562 generic.go:356] "Generic (PLEG): container finished" podID="473ad98e-110c-4d09-a2c6-ac1655312dd8" containerID="3e8580cc1e3bc137669ce6b20fca50ef322a1d69970d0d4ce45ecdda6f55f52f" exitCode=0
Sep 16 18:16:40.471099 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.471070    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnngf" event={"ID":"473ad98e-110c-4d09-a2c6-ac1655312dd8","Type":"ContainerDied","Data":"3e8580cc1e3bc137669ce6b20fca50ef322a1d69970d0d4ce45ecdda6f55f52f"}
Sep 16 18:16:40.472432 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.472415    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9fbvc" event={"ID":"c391df85-454c-4e29-83e5-32a677d165d2","Type":"ContainerStarted","Data":"836e7b0aa95c602521762aae6ba89b85354aefcce10d1f7b9710f944235f5c2f"}
Sep 16 18:16:40.473884 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.473858    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n9h6f" event={"ID":"a81ed7be-fe2e-49d3-9dd8-0aa0511aaaa4","Type":"ContainerStarted","Data":"262ef65d1ff8f23540bc30dfce8aacdd550b43f344b145b2344f18e3da9ee3b4"}
Sep 16 18:16:40.475109 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.475094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fxbpm" event={"ID":"634753ab-bf55-4898-a91d-7b08037e8ac3","Type":"ContainerStarted","Data":"ce4f026616ae4bb3484f4e506b45a577640385e707911291c38cb12afd9d03d2"}
Sep 16 18:16:40.481279 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.481237    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gzdnv" podStartSLOduration=2.48428554 podStartE2EDuration="20.481226863s" totalImagesPullingTime="17.996941323s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:40.480089203 +0000 UTC m=+20.719877956" watchObservedRunningTime="2026-09-16 18:16:40.481226863 +0000 UTC m=+20.721015615"
Sep 16 18:16:40.500291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.500259    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9fbvc" podStartSLOduration=2.521251562 podStartE2EDuration="20.500249441s" totalImagesPullingTime="17.978997879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:40.499871012 +0000 UTC m=+20.739659763" watchObservedRunningTime="2026-09-16 18:16:40.500249441 +0000 UTC m=+20.740038192"
Sep 16 18:16:40.538808 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.538774    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" podStartSLOduration=19.538762754 podStartE2EDuration="19.538762754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:40.53823873 +0000 UTC m=+20.778027493" watchObservedRunningTime="2026-09-16 18:16:40.538762754 +0000 UTC m=+20.778551548"
Sep 16 18:16:40.555203 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.554656    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4wchr" podStartSLOduration=2.50535186 podStartE2EDuration="20.554641685s" totalImagesPullingTime="18.049289825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:40.554207115 +0000 UTC m=+20.793995867" watchObservedRunningTime="2026-09-16 18:16:40.554641685 +0000 UTC m=+20.794430437"
Sep 16 18:16:40.569264 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.569230    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fxbpm" podStartSLOduration=6.588737167 podStartE2EDuration="20.569222541s" totalImagesPullingTime="13.980485374s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:40.56919531 +0000 UTC m=+20.808984061" watchObservedRunningTime="2026-09-16 18:16:40.569222541 +0000 UTC m=+20.809011293"
Sep 16 18:16:40.590422 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:40.590380    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n9h6f" podStartSLOduration=2.292800379 podStartE2EDuration="20.590366529s" totalImagesPullingTime="18.29756615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:40.59034049 +0000 UTC m=+20.830129245" watchObservedRunningTime="2026-09-16 18:16:40.590366529 +0000 UTC m=+20.830155278"
Sep 16 18:16:41.279656 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.279628    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:41.279839 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.279628    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:41.279839 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:41.279741    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:41.279839 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.279628    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:41.280027 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:41.279835    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:41.280027 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:41.279932    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:41.405023 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.404997    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:41.405771 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.405753    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:41.479987 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.479955    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" event={"ID":"daaf67e5c8e04e118d334be2e4278536","Type":"ContainerStarted","Data":"613a6a323e0b96170d02688de02c912bcbb316691ae198c5cce9da9d416961e9"}
Sep 16 18:16:41.481673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.481651    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c4vt7" event={"ID":"892f1cab-d005-4df0-ac49-799b3445cc57","Type":"ContainerStarted","Data":"526a71ffc38175df19c63214767ab7226dbae93acdba186423fb79342d83e571"}
Sep 16 18:16:41.485634 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.485611    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h" event={"ID":"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a","Type":"ContainerStarted","Data":"777aeb8520f93dc26d0f21258a849bc5d7ff1b317fc37ff2c38ed10a32a25859"}
Sep 16 18:16:41.485727 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.485668    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h" event={"ID":"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a","Type":"ContainerStarted","Data":"4dafa298a324c5ce83c319c1d7abbd458ba9393dc07f92fed86081de8f67323f"}
Sep 16 18:16:41.498623 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.498560    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" podStartSLOduration=20.498548473 podStartE2EDuration="20.498548473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:41.498060708 +0000 UTC m=+21.737849459" watchObservedRunningTime="2026-09-16 18:16:41.498548473 +0000 UTC m=+21.738337216"
Sep 16 18:16:41.517811 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.517630    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c4vt7" podStartSLOduration=3.4844542880000002 podStartE2EDuration="21.517614798s" totalImagesPullingTime="18.03316051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:41.517168773 +0000 UTC m=+21.756957524" watchObservedRunningTime="2026-09-16 18:16:41.517614798 +0000 UTC m=+21.757403551"
Sep 16 18:16:41.543004 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:41.542969    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 18:16:42.214044 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:42.213938    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T18:16:41.542982735Z","UUID":"52138c77-2140-42e3-906a-db4254faf87c","Handler":null,"Name":"","Endpoint":""}
Sep 16 18:16:42.216074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:42.216051    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 18:16:42.216228 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:42.216086    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 18:16:42.490564 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:42.490457    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8" event={"ID":"59507886-85d1-4a81-b563-59ef4dd70fb6","Type":"ContainerStarted","Data":"374ad44f3692bc1d6192e2df7e151b1e3b6e4478e83a0e2d952fc18d2a0b836c"}
Sep 16 18:16:42.490564 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:42.490518    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 18:16:43.280182 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:43.280144    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:43.280415 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:43.280150    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:43.280415 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:43.280263    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:43.280415 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:43.280144    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:43.280415 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:43.280310    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:43.280612 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:43.280449    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:43.496895 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:43.496862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h" event={"ID":"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a","Type":"ContainerStarted","Data":"3b2e7654b119a2929785b5212babdcb1e442599dcac7491eadb0599de7ddd123"}
Sep 16 18:16:43.499554 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:43.499520    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8" event={"ID":"59507886-85d1-4a81-b563-59ef4dd70fb6","Type":"ContainerStarted","Data":"c79790475365917b7b39b4fd9c920969b2137ae715bc6df0b6f6794807488b13"}
Sep 16 18:16:43.523258 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:43.523211    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwbp8" podStartSLOduration=3.267065214 podStartE2EDuration="23.523192324s" totalImagesPullingTime="20.25612711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:43.52213882 +0000 UTC m=+23.761927574" watchObservedRunningTime="2026-09-16 18:16:43.523192324 +0000 UTC m=+23.762981077"
Sep 16 18:16:45.280124 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:45.279988    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:45.280450 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:45.279992    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:45.280450 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:45.280198    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:45.280450 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:45.280026    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:45.280450 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:45.280276    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:45.280450 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:45.280328    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:45.505680 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:45.505650    2562 generic.go:356] "Generic (PLEG): container finished" podID="473ad98e-110c-4d09-a2c6-ac1655312dd8" containerID="77c8a9c4ec20fbfb7a947fae965738ada7064d4f3a6f171674f2cd0c461db380" exitCode=0
Sep 16 18:16:45.505880 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:45.505725    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnngf" event={"ID":"473ad98e-110c-4d09-a2c6-ac1655312dd8","Type":"ContainerDied","Data":"77c8a9c4ec20fbfb7a947fae965738ada7064d4f3a6f171674f2cd0c461db380"}
Sep 16 18:16:45.510070 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:45.510048    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h" event={"ID":"6ae2c74c-dbdb-46f2-8c29-e626d3cc052a","Type":"ContainerStarted","Data":"5e457dcb44de5d0f0d96f572879b137d2dc17aeb9c77eb6ee554b859a9de3beb"}
Sep 16 18:16:45.510314 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:45.510297    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:45.510379 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:45.510355    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:45.525667 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:45.525649    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:45.558671 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:45.558578    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h" podStartSLOduration=7.260767333 podStartE2EDuration="25.558567348s" totalImagesPullingTime="18.297800015s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:45.558377691 +0000 UTC m=+25.798166442" watchObservedRunningTime="2026-09-16 18:16:45.558567348 +0000 UTC m=+25.798356099"
Sep 16 18:16:46.514905 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:46.514826    2562 generic.go:356] "Generic (PLEG): container finished" podID="473ad98e-110c-4d09-a2c6-ac1655312dd8" containerID="34d56106caeb15518dd904aefb579ff3bccee9d77d7918335bea594236826f6d" exitCode=0
Sep 16 18:16:46.515661 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:46.514900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnngf" event={"ID":"473ad98e-110c-4d09-a2c6-ac1655312dd8","Type":"ContainerDied","Data":"34d56106caeb15518dd904aefb579ff3bccee9d77d7918335bea594236826f6d"}
Sep 16 18:16:46.515661 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:46.515350    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:46.533544 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:46.533521    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:16:46.750094 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:46.750069    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2x4vg"]
Sep 16 18:16:46.750205 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:46.750172    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:46.750271 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:46.750256    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:46.753460 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:46.753438    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mp8m7"]
Sep 16 18:16:46.753584 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:46.753537    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:46.753684 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:46.753664    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:46.754119 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:46.754095    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p8h5l"]
Sep 16 18:16:46.754200 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:46.754175    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:46.754256 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:46.754242    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:48.279392 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:48.279195    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:48.279692 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:48.279200    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:48.279692 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:48.279486    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:48.279692 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:48.279226    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:48.279692 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:48.279541    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:48.279692 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:48.279659    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:48.522088 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:48.522069    2562 generic.go:356] "Generic (PLEG): container finished" podID="473ad98e-110c-4d09-a2c6-ac1655312dd8" containerID="2c3475182b4ba260e9fd637725ae1bfa218d86ad4fd5b8a7b6f02daf6abe2089" exitCode=0
Sep 16 18:16:48.522193 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:48.522133    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnngf" event={"ID":"473ad98e-110c-4d09-a2c6-ac1655312dd8","Type":"ContainerDied","Data":"2c3475182b4ba260e9fd637725ae1bfa218d86ad4fd5b8a7b6f02daf6abe2089"}
Sep 16 18:16:50.282759 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:50.282732    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:50.283168 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:50.282834    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:50.283168 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:50.282841    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:50.283168 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:50.282858    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:50.283168 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:50.282943    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:50.283168 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:50.283008    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:51.316375 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:51.316345    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:51.316842 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:51.316471    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 18:16:51.317012 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:51.316996    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4wchr"
Sep 16 18:16:51.437811 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:51.437772    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:16:51.570260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:51.570177    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:16:52.280021 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.279993    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:52.280021 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.280021    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:52.280232 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.279993    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:52.280232 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:52.280115    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp8m7" podUID="c0001fd5-b780-42e5-a8f8-84f1f01076fe"
Sep 16 18:16:52.280232 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:52.280166    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2x4vg" podUID="8b056af3-46bd-4340-8336-66d3e52a40f7"
Sep 16 18:16:52.280368 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:52.280271    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p8h5l" podUID="9594a6fd-1697-4d79-a706-500845eb00f8"
Sep 16 18:16:52.534503 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.534442    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeReady"
Sep 16 18:16:52.534913 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.534555    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 18:16:52.581014 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.580992    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4dzgz"]
Sep 16 18:16:52.590012 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.589991    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q4tb8"]
Sep 16 18:16:52.590120 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.590103    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.592473 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.592458    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 18:16:52.592562 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.592530    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 18:16:52.592562 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.592552    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f8sdc\""
Sep 16 18:16:52.600253 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.600235    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4dzgz"]
Sep 16 18:16:52.600332 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.600265    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q4tb8"]
Sep 16 18:16:52.600332 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.600320    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q4tb8"
Sep 16 18:16:52.602533 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.602513    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 18:16:52.602632 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.602565    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zhggq\""
Sep 16 18:16:52.602791 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.602775    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 18:16:52.602857 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.602794    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 18:16:52.710637 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.710619    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt2b5\" (UniqueName: \"kubernetes.io/projected/1ea3df26-130f-4aeb-a1e9-527f755670d5-kube-api-access-mt2b5\") pod \"ingress-canary-q4tb8\" (UID: \"1ea3df26-130f-4aeb-a1e9-527f755670d5\") " pod="openshift-ingress-canary/ingress-canary-q4tb8"
Sep 16 18:16:52.710722 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.710643    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ea3df26-130f-4aeb-a1e9-527f755670d5-cert\") pod \"ingress-canary-q4tb8\" (UID: \"1ea3df26-130f-4aeb-a1e9-527f755670d5\") " pod="openshift-ingress-canary/ingress-canary-q4tb8"
Sep 16 18:16:52.710722 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.710660    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc29fcde-2de7-4474-beff-440b1e1724cf-tmp-dir\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.710722 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.710673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv6tq\" (UniqueName: \"kubernetes.io/projected/bc29fcde-2de7-4474-beff-440b1e1724cf-kube-api-access-kv6tq\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.710819 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.710735    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc29fcde-2de7-4474-beff-440b1e1724cf-config-volume\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.710819 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.710780    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.811259 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.811219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc29fcde-2de7-4474-beff-440b1e1724cf-config-volume\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.811259 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.811258    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.811358 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.811286    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt2b5\" (UniqueName: \"kubernetes.io/projected/1ea3df26-130f-4aeb-a1e9-527f755670d5-kube-api-access-mt2b5\") pod \"ingress-canary-q4tb8\" (UID: \"1ea3df26-130f-4aeb-a1e9-527f755670d5\") " pod="openshift-ingress-canary/ingress-canary-q4tb8"
Sep 16 18:16:52.811358 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.811304    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ea3df26-130f-4aeb-a1e9-527f755670d5-cert\") pod \"ingress-canary-q4tb8\" (UID: \"1ea3df26-130f-4aeb-a1e9-527f755670d5\") " pod="openshift-ingress-canary/ingress-canary-q4tb8"
Sep 16 18:16:52.811436 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:52.811365    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:16:52.811436 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:52.811413    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls podName:bc29fcde-2de7-4474-beff-440b1e1724cf nodeName:}" failed. No retries permitted until 2026-09-16 18:16:53.31140072 +0000 UTC m=+33.551189450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls") pod "dns-default-4dzgz" (UID: "bc29fcde-2de7-4474-beff-440b1e1724cf") : secret "dns-default-metrics-tls" not found
Sep 16 18:16:52.811513 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.811474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc29fcde-2de7-4474-beff-440b1e1724cf-tmp-dir\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.811513 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.811503    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv6tq\" (UniqueName: \"kubernetes.io/projected/bc29fcde-2de7-4474-beff-440b1e1724cf-kube-api-access-kv6tq\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.811747 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.811732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc29fcde-2de7-4474-beff-440b1e1724cf-config-volume\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.811793 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.811749    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc29fcde-2de7-4474-beff-440b1e1724cf-tmp-dir\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.815330 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.815307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ea3df26-130f-4aeb-a1e9-527f755670d5-cert\") pod \"ingress-canary-q4tb8\" (UID: \"1ea3df26-130f-4aeb-a1e9-527f755670d5\") " pod="openshift-ingress-canary/ingress-canary-q4tb8"
Sep 16 18:16:52.821850 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.821826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv6tq\" (UniqueName: \"kubernetes.io/projected/bc29fcde-2de7-4474-beff-440b1e1724cf-kube-api-access-kv6tq\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:52.822019 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.822002    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt2b5\" (UniqueName: \"kubernetes.io/projected/1ea3df26-130f-4aeb-a1e9-527f755670d5-kube-api-access-mt2b5\") pod \"ingress-canary-q4tb8\" (UID: \"1ea3df26-130f-4aeb-a1e9-527f755670d5\") " pod="openshift-ingress-canary/ingress-canary-q4tb8"
Sep 16 18:16:52.908829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.908715    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q4tb8"
Sep 16 18:16:52.911967 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.911945    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjmc\" (UniqueName: \"kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc\") pod \"network-check-target-p8h5l\" (UID: \"9594a6fd-1697-4d79-a706-500845eb00f8\") " pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:52.912053 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:52.911987    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:52.912089 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:52.912055    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:52.912119 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:52.912084    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:52.912119 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:52.912101    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:52.912119 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:52.912106    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret podName:8b056af3-46bd-4340-8336-66d3e52a40f7 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:24.912094119 +0000 UTC m=+65.151882849 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret") pod "global-pull-secret-syncer-2x4vg" (UID: "8b056af3-46bd-4340-8336-66d3e52a40f7") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:52.912119 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:52.912110    2562 projected.go:196] Error preparing data for projected volume kube-api-access-kqjmc for pod openshift-network-diagnostics/network-check-target-p8h5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:52.912257 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:52.912177    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc podName:9594a6fd-1697-4d79-a706-500845eb00f8 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:24.912165528 +0000 UTC m=+65.151954259 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqjmc" (UniqueName: "kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc") pod "network-check-target-p8h5l" (UID: "9594a6fd-1697-4d79-a706-500845eb00f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:53.013157 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:53.013131    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:53.013293 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:53.013242    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:53.013293 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:53.013288    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs podName:c0001fd5-b780-42e5-a8f8-84f1f01076fe nodeName:}" failed. No retries permitted until 2026-09-16 18:17:25.013274551 +0000 UTC m=+65.253063281 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs") pod "network-metrics-daemon-mp8m7" (UID: "c0001fd5-b780-42e5-a8f8-84f1f01076fe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:53.230865 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:53.230835    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q4tb8"]
Sep 16 18:16:53.234343 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:16:53.234318    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea3df26_130f_4aeb_a1e9_527f755670d5.slice/crio-f8f4238b7a0836f092dd2a2e55add085fd061a4ee2c0283eab32181088af1b70 WatchSource:0}: Error finding container f8f4238b7a0836f092dd2a2e55add085fd061a4ee2c0283eab32181088af1b70: Status 404 returned error can't find the container with id f8f4238b7a0836f092dd2a2e55add085fd061a4ee2c0283eab32181088af1b70
Sep 16 18:16:53.314360 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:53.314337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:16:53.315181 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:53.315165    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:53.315264 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:53.315253    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:16:53.315306 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:53.315298    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls podName:bc29fcde-2de7-4474-beff-440b1e1724cf nodeName:}" failed. No retries permitted until 2026-09-16 18:16:54.31528661 +0000 UTC m=+34.555075340 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls") pod "dns-default-4dzgz" (UID: "bc29fcde-2de7-4474-beff-440b1e1724cf") : secret "dns-default-metrics-tls" not found
Sep 16 18:16:53.377820 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:53.377765    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:16:53.539307 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:53.539283    2562 generic.go:356] "Generic (PLEG): container finished" podID="473ad98e-110c-4d09-a2c6-ac1655312dd8" containerID="7a1476e5076b9605166cac31114ca29e550e515761e0a911a7f06fe488ced8ed" exitCode=0
Sep 16 18:16:53.539871 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:53.539360    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnngf" event={"ID":"473ad98e-110c-4d09-a2c6-ac1655312dd8","Type":"ContainerDied","Data":"7a1476e5076b9605166cac31114ca29e550e515761e0a911a7f06fe488ced8ed"}
Sep 16 18:16:53.540556 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:53.540534    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q4tb8" event={"ID":"1ea3df26-130f-4aeb-a1e9-527f755670d5","Type":"ContainerStarted","Data":"f8f4238b7a0836f092dd2a2e55add085fd061a4ee2c0283eab32181088af1b70"}
Sep 16 18:16:54.279671 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.279644    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:16:54.279912 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.279768    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:16:54.280093 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.280058    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:16:54.283808 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.283792    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:16:54.284163 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.284133    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bfgtt\""
Sep 16 18:16:54.285206 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.284928    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:16:54.285206 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.284943    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:16:54.285206 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.284944    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5rgdl\""
Sep 16 18:16:54.285206 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.284999    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:16:54.323600 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.323559    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:54.323709 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:54.323689    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:16:54.323778 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:54.323766    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls podName:bc29fcde-2de7-4474-beff-440b1e1724cf nodeName:}" failed. No retries permitted until 2026-09-16 18:16:56.323747513 +0000 UTC m=+36.563536246 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls") pod "dns-default-4dzgz" (UID: "bc29fcde-2de7-4474-beff-440b1e1724cf") : secret "dns-default-metrics-tls" not found
Sep 16 18:16:54.547003 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.546918    2562 generic.go:356] "Generic (PLEG): container finished" podID="473ad98e-110c-4d09-a2c6-ac1655312dd8" containerID="51c87f08084363efdcbd9011ce7fc20a294110dc7daa7bcec1e72dc770919bdf" exitCode=0
Sep 16 18:16:54.547527 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:54.547016    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnngf" event={"ID":"473ad98e-110c-4d09-a2c6-ac1655312dd8","Type":"ContainerDied","Data":"51c87f08084363efdcbd9011ce7fc20a294110dc7daa7bcec1e72dc770919bdf"}
Sep 16 18:16:55.552310 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:55.552271    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xnngf" event={"ID":"473ad98e-110c-4d09-a2c6-ac1655312dd8","Type":"ContainerStarted","Data":"392c174a4e80fd3444ac8486afca0a41188f7e243e0e5febeaf967219dbd717c"}
Sep 16 18:16:55.577714 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:55.577666    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xnngf" podStartSLOduration=7.302736388 podStartE2EDuration="35.57764867s" totalImagesPullingTime="28.274912282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:55.577157004 +0000 UTC m=+35.816945755" watchObservedRunningTime="2026-09-16 18:16:55.57764867 +0000 UTC m=+35.817437422"
Sep 16 18:16:55.700805 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:55.700662    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:16:55.771379 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:55.771354    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:16:56.313152 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:56.313121    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:16:56.340059 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:56.340032    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:16:56.340156 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:56.340134    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:16:56.340193 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:16:56.340179    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls podName:bc29fcde-2de7-4474-beff-440b1e1724cf nodeName:}" failed. No retries permitted until 2026-09-16 18:17:00.340167785 +0000 UTC m=+40.579956515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls") pod "dns-default-4dzgz" (UID: "bc29fcde-2de7-4474-beff-440b1e1724cf") : secret "dns-default-metrics-tls" not found
Sep 16 18:16:56.414580 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:56.414558    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:16:56.556408 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:56.556330    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q4tb8" event={"ID":"1ea3df26-130f-4aeb-a1e9-527f755670d5","Type":"ContainerStarted","Data":"930e578d95dc87880315d2d45d167c6bbadb5af4ed44670e4465f26099f393ae"}
Sep 16 18:16:56.573775 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:56.573733    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q4tb8" podStartSLOduration=2.036680056 podStartE2EDuration="4.573719867s" totalImagesPullingTime="2.537039811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:16:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:56.573251064 +0000 UTC m=+36.813039817" watchObservedRunningTime="2026-09-16 18:16:56.573719867 +0000 UTC m=+36.813508617"
Sep 16 18:16:56.656627 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:56.656580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:16:56.716472 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:56.716447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:16:57.946825 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:57.946789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:16:58.014605 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:16:58.014572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:00.367208 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:00.367174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:17:00.367547 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:00.367311    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:17:00.367547 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:00.367370    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls podName:bc29fcde-2de7-4474-beff-440b1e1724cf nodeName:}" failed. No retries permitted until 2026-09-16 18:17:08.36735753 +0000 UTC m=+48.607146260 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls") pod "dns-default-4dzgz" (UID: "bc29fcde-2de7-4474-beff-440b1e1724cf") : secret "dns-default-metrics-tls" not found
Sep 16 18:17:08.417003 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:08.416971    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:17:08.417413 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:08.417108    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:17:08.417413 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:08.417172    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls podName:bc29fcde-2de7-4474-beff-440b1e1724cf nodeName:}" failed. No retries permitted until 2026-09-16 18:17:24.417153608 +0000 UTC m=+64.656942340 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls") pod "dns-default-4dzgz" (UID: "bc29fcde-2de7-4474-beff-440b1e1724cf") : secret "dns-default-metrics-tls" not found
Sep 16 18:17:09.646418 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:09.646385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:09.746887 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:09.746849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:17.106757 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.106727    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-qnrm5"]
Sep 16 18:17:17.108876 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.108859    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-qnrm5"
Sep 16 18:17:17.111696 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.111668    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-wnv8c\""
Sep 16 18:17:17.111946 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.111933    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:17:17.112529 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.112510    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 16 18:17:17.125745 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.125725    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7fcd5855b4-c2lsw"]
Sep 16 18:17:17.130161 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.130136    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-qnrm5"]
Sep 16 18:17:17.130245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.130211    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.140606 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.140575    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 18:17:17.140795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.140783    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 16 18:17:17.140977 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.140957    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 18:17:17.141079 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.141005    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 16 18:17:17.141351 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.141333    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 16 18:17:17.141552 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.141538    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 16 18:17:17.142074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.142059    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-85f6b\""
Sep 16 18:17:17.149394 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.149374    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7fcd5855b4-c2lsw"]
Sep 16 18:17:17.217392 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.217365    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"]
Sep 16 18:17:17.218944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.218921    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"
Sep 16 18:17:17.221290 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.221273    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 16 18:17:17.221394 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.221379    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 16 18:17:17.221845 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.221830    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2nw2k\""
Sep 16 18:17:17.222019 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.222007    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:17:17.222451 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.222440    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 16 18:17:17.223945 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.223920    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"]
Sep 16 18:17:17.225641 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.225624    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-42dj7"]
Sep 16 18:17:17.225742 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.225720    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"
Sep 16 18:17:17.227004 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.226989    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.230867 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.230850    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 18:17:17.231530 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.231254    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 16 18:17:17.231530 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.231464    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 18:17:17.231872 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.231858    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 16 18:17:17.231975 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.231959    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zvr25\""
Sep 16 18:17:17.232081 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.232066    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8mmct\""
Sep 16 18:17:17.232171 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.232155    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 16 18:17:17.232619 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.232602    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:17:17.235670 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.235655    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 16 18:17:17.236093 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.236073    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 16 18:17:17.240107 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.240087    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"]
Sep 16 18:17:17.241852 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.241837    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 16 18:17:17.243801 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.243785    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"]
Sep 16 18:17:17.244675 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.244656    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-42dj7"]
Sep 16 18:17:17.274155 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.274138    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-default-certificate\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.274263 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.274176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.274263 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.274200    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-stats-auth\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.274263 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.274215    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.274263 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.274256    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmpj7\" (UniqueName: \"kubernetes.io/projected/64086446-7f12-484e-b1e0-c0f57190258e-kube-api-access-xmpj7\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.274398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.274297    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhjgx\" (UniqueName: \"kubernetes.io/projected/6f83ab39-0687-4870-a4d6-2cb1aaf4a0e1-kube-api-access-dhjgx\") pod \"volume-data-source-validator-86695b7cc7-qnrm5\" (UID: \"6f83ab39-0687-4870-a4d6-2cb1aaf4a0e1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-qnrm5"
Sep 16 18:17:17.353447 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.353404    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:17.375286 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375236    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5eb43333-facd-4124-8053-8008cd52f251-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xw4vh\" (UID: \"5eb43333-facd-4124-8053-8008cd52f251\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"
Sep 16 18:17:17.375286 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-stats-auth\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.375286 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375290    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e3a2033-5283-49f6-8407-ad0a2fe0a1d3-serving-cert\") pod \"service-ca-operator-cf7c776b7-hh4gp\" (UID: \"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"
Sep 16 18:17:17.375462 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375306    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.375462 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375321    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmpj7\" (UniqueName: \"kubernetes.io/projected/64086446-7f12-484e-b1e0-c0f57190258e-kube-api-access-xmpj7\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.375462 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375336    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhjgx\" (UniqueName: \"kubernetes.io/projected/6f83ab39-0687-4870-a4d6-2cb1aaf4a0e1-kube-api-access-dhjgx\") pod \"volume-data-source-validator-86695b7cc7-qnrm5\" (UID: \"6f83ab39-0687-4870-a4d6-2cb1aaf4a0e1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-qnrm5"
Sep 16 18:17:17.375462 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375350    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56254d14-1b92-4b27-aa73-678c951f78ef-service-ca-bundle\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.375462 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375386    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e3a2033-5283-49f6-8407-ad0a2fe0a1d3-config\") pod \"service-ca-operator-cf7c776b7-hh4gp\" (UID: \"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"
Sep 16 18:17:17.375462 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:17.375388    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:17:17.375753 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:17.375467    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:17.875451472 +0000 UTC m=+58.115240222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : secret "router-metrics-certs-default" not found
Sep 16 18:17:17.375753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375495    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-default-certificate\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.375753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375516    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfxgl\" (UniqueName: \"kubernetes.io/projected/5eb43333-facd-4124-8053-8008cd52f251-kube-api-access-cfxgl\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xw4vh\" (UID: \"5eb43333-facd-4124-8053-8008cd52f251\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"
Sep 16 18:17:17.375753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375533    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/56254d14-1b92-4b27-aa73-678c951f78ef-serving-cert\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.375753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375561    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/56254d14-1b92-4b27-aa73-678c951f78ef-snapshots\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.375753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375603    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5eb43333-facd-4124-8053-8008cd52f251-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xw4vh\" (UID: \"5eb43333-facd-4124-8053-8008cd52f251\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"
Sep 16 18:17:17.375753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375622    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtgjb\" (UniqueName: \"kubernetes.io/projected/56254d14-1b92-4b27-aa73-678c951f78ef-kube-api-access-xtgjb\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.375753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375702    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.376030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375750    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56254d14-1b92-4b27-aa73-678c951f78ef-tmp\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.376030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375785    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56254d14-1b92-4b27-aa73-678c951f78ef-trusted-ca-bundle\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.376030 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:17.375800    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:17.875789915 +0000 UTC m=+58.115578646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : configmap references non-existent config key: service-ca.crt
Sep 16 18:17:17.376030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.375832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmtbp\" (UniqueName: \"kubernetes.io/projected/8e3a2033-5283-49f6-8407-ad0a2fe0a1d3-kube-api-access-xmtbp\") pod \"service-ca-operator-cf7c776b7-hh4gp\" (UID: \"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"
Sep 16 18:17:17.378842 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.378824    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-stats-auth\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.378931 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.378832    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-default-certificate\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.388995 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.388976    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhjgx\" (UniqueName: \"kubernetes.io/projected/6f83ab39-0687-4870-a4d6-2cb1aaf4a0e1-kube-api-access-dhjgx\") pod \"volume-data-source-validator-86695b7cc7-qnrm5\" (UID: \"6f83ab39-0687-4870-a4d6-2cb1aaf4a0e1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-qnrm5"
Sep 16 18:17:17.389099 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.389081    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmpj7\" (UniqueName: \"kubernetes.io/projected/64086446-7f12-484e-b1e0-c0f57190258e-kube-api-access-xmpj7\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.416847 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.416821    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-qnrm5"
Sep 16 18:17:17.450214 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.450188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:17.477027 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477003    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56254d14-1b92-4b27-aa73-678c951f78ef-tmp\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.477158 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477046    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56254d14-1b92-4b27-aa73-678c951f78ef-trusted-ca-bundle\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.477158 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477075    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmtbp\" (UniqueName: \"kubernetes.io/projected/8e3a2033-5283-49f6-8407-ad0a2fe0a1d3-kube-api-access-xmtbp\") pod \"service-ca-operator-cf7c776b7-hh4gp\" (UID: \"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"
Sep 16 18:17:17.477158 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477107    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5eb43333-facd-4124-8053-8008cd52f251-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xw4vh\" (UID: \"5eb43333-facd-4124-8053-8008cd52f251\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"
Sep 16 18:17:17.477158 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e3a2033-5283-49f6-8407-ad0a2fe0a1d3-serving-cert\") pod \"service-ca-operator-cf7c776b7-hh4gp\" (UID: \"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"
Sep 16 18:17:17.477368 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56254d14-1b92-4b27-aa73-678c951f78ef-service-ca-bundle\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.477368 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e3a2033-5283-49f6-8407-ad0a2fe0a1d3-config\") pod \"service-ca-operator-cf7c776b7-hh4gp\" (UID: \"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"
Sep 16 18:17:17.477368 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfxgl\" (UniqueName: \"kubernetes.io/projected/5eb43333-facd-4124-8053-8008cd52f251-kube-api-access-cfxgl\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xw4vh\" (UID: \"5eb43333-facd-4124-8053-8008cd52f251\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"
Sep 16 18:17:17.477368 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477294    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/56254d14-1b92-4b27-aa73-678c951f78ef-serving-cert\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.477368 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477331    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/56254d14-1b92-4b27-aa73-678c951f78ef-snapshots\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.477368 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5eb43333-facd-4124-8053-8008cd52f251-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xw4vh\" (UID: \"5eb43333-facd-4124-8053-8008cd52f251\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"
Sep 16 18:17:17.477688 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtgjb\" (UniqueName: \"kubernetes.io/projected/56254d14-1b92-4b27-aa73-678c951f78ef-kube-api-access-xtgjb\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.477688 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.477495    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56254d14-1b92-4b27-aa73-678c951f78ef-tmp\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.479967 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.478077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/56254d14-1b92-4b27-aa73-678c951f78ef-snapshots\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.479967 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.478473    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5eb43333-facd-4124-8053-8008cd52f251-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xw4vh\" (UID: \"5eb43333-facd-4124-8053-8008cd52f251\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"
Sep 16 18:17:17.479967 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.478486    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e3a2033-5283-49f6-8407-ad0a2fe0a1d3-config\") pod \"service-ca-operator-cf7c776b7-hh4gp\" (UID: \"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"
Sep 16 18:17:17.479967 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.479251    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56254d14-1b92-4b27-aa73-678c951f78ef-trusted-ca-bundle\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.479967 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.479692    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56254d14-1b92-4b27-aa73-678c951f78ef-service-ca-bundle\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.480800 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.480781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e3a2033-5283-49f6-8407-ad0a2fe0a1d3-serving-cert\") pod \"service-ca-operator-cf7c776b7-hh4gp\" (UID: \"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"
Sep 16 18:17:17.481719 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.481703    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5eb43333-facd-4124-8053-8008cd52f251-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xw4vh\" (UID: \"5eb43333-facd-4124-8053-8008cd52f251\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"
Sep 16 18:17:17.481936 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.481922    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/56254d14-1b92-4b27-aa73-678c951f78ef-serving-cert\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.486643 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.486625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfxgl\" (UniqueName: \"kubernetes.io/projected/5eb43333-facd-4124-8053-8008cd52f251-kube-api-access-cfxgl\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xw4vh\" (UID: \"5eb43333-facd-4124-8053-8008cd52f251\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"
Sep 16 18:17:17.489753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.489731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtgjb\" (UniqueName: \"kubernetes.io/projected/56254d14-1b92-4b27-aa73-678c951f78ef-kube-api-access-xtgjb\") pod \"insights-operator-5844b856db-42dj7\" (UID: \"56254d14-1b92-4b27-aa73-678c951f78ef\") " pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.490819 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.490804    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmtbp\" (UniqueName: \"kubernetes.io/projected/8e3a2033-5283-49f6-8407-ad0a2fe0a1d3-kube-api-access-xmtbp\") pod \"service-ca-operator-cf7c776b7-hh4gp\" (UID: \"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"
Sep 16 18:17:17.526801 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.526783    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"
Sep 16 18:17:17.534367 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.534342    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"
Sep 16 18:17:17.538369 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.538353    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-42dj7"
Sep 16 18:17:17.684283 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.684258    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-qnrm5"]
Sep 16 18:17:17.687411 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:17:17.687387    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f83ab39_0687_4870_a4d6_2cb1aaf4a0e1.slice/crio-bae5a03b72d617194bf32710f7ac66bca291dfe2c07ed393533b58a90b7c26d8 WatchSource:0}: Error finding container bae5a03b72d617194bf32710f7ac66bca291dfe2c07ed393533b58a90b7c26d8: Status 404 returned error can't find the container with id bae5a03b72d617194bf32710f7ac66bca291dfe2c07ed393533b58a90b7c26d8
Sep 16 18:17:17.800699 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.800665    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp"]
Sep 16 18:17:17.823040 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.823020    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh"]
Sep 16 18:17:17.825662 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:17:17.825640    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb43333_facd_4124_8053_8008cd52f251.slice/crio-f0856119e142d9c7c3ff6dfb2a072627010c82e22d785c19affbda25ec36fedf WatchSource:0}: Error finding container f0856119e142d9c7c3ff6dfb2a072627010c82e22d785c19affbda25ec36fedf: Status 404 returned error can't find the container with id f0856119e142d9c7c3ff6dfb2a072627010c82e22d785c19affbda25ec36fedf
Sep 16 18:17:17.826434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.826416    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-42dj7"]
Sep 16 18:17:17.828425 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:17:17.828397    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56254d14_1b92_4b27_aa73_678c951f78ef.slice/crio-7d1bae62acd0df83cc7784725b141191f5710652f592f406fbf6e229e69134d9 WatchSource:0}: Error finding container 7d1bae62acd0df83cc7784725b141191f5710652f592f406fbf6e229e69134d9: Status 404 returned error can't find the container with id 7d1bae62acd0df83cc7784725b141191f5710652f592f406fbf6e229e69134d9
Sep 16 18:17:17.881318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.881293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.881404 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:17.881363    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:17.881461 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:17.881443    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:17:17.881497 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:17.881466    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:18.881446799 +0000 UTC m=+59.121235544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : configmap references non-existent config key: service-ca.crt
Sep 16 18:17:17.881497 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:17.881489    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:18.881478172 +0000 UTC m=+59.121266907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : secret "router-metrics-certs-default" not found
Sep 16 18:17:18.536875 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:18.536333    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c2w7h"
Sep 16 18:17:18.608760 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:18.608725    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-42dj7" event={"ID":"56254d14-1b92-4b27-aa73-678c951f78ef","Type":"ContainerStarted","Data":"7d1bae62acd0df83cc7784725b141191f5710652f592f406fbf6e229e69134d9"}
Sep 16 18:17:18.611249 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:18.611158    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh" event={"ID":"5eb43333-facd-4124-8053-8008cd52f251","Type":"ContainerStarted","Data":"f0856119e142d9c7c3ff6dfb2a072627010c82e22d785c19affbda25ec36fedf"}
Sep 16 18:17:18.612686 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:18.612662    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-qnrm5" event={"ID":"6f83ab39-0687-4870-a4d6-2cb1aaf4a0e1","Type":"ContainerStarted","Data":"bae5a03b72d617194bf32710f7ac66bca291dfe2c07ed393533b58a90b7c26d8"}
Sep 16 18:17:18.616342 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:18.616317    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp" event={"ID":"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3","Type":"ContainerStarted","Data":"c86a130e6eda9a3ed949db72ad67a6c8e3e066bc57c922e5129b185eba30323f"}
Sep 16 18:17:18.701929 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:18.701889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:18.761653 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:18.761447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:18.889321 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:18.888672    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:18.889321 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:18.888793    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:18.889321 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:18.888949    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:20.888929864 +0000 UTC m=+61.128718596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : configmap references non-existent config key: service-ca.crt
Sep 16 18:17:18.889654 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:18.889341    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:17:18.889654 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:18.889391    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:20.88937572 +0000 UTC m=+61.129164465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : secret "router-metrics-certs-default" not found
Sep 16 18:17:19.621396 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:19.621351    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-qnrm5" event={"ID":"6f83ab39-0687-4870-a4d6-2cb1aaf4a0e1","Type":"ContainerStarted","Data":"d16d4053751657b9fe6ee58371a7effcacc6255818360c409fafe862fcd2bd28"}
Sep 16 18:17:19.638879 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:19.638827    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-qnrm5" podStartSLOduration=1.272635132 podStartE2EDuration="2.638805737s" totalImagesPullingTime="1.366170605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:19.638093386 +0000 UTC m=+59.877882140" watchObservedRunningTime="2026-09-16 18:17:19.638805737 +0000 UTC m=+59.878594488"
Sep 16 18:17:19.747246 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:19.747204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:19.807794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:19.807754    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:20.906830 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:20.906670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:20.906830 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:20.906719    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:20.906830 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:20.906808    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:17:20.907298 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:20.906852    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:24.906839535 +0000 UTC m=+65.146628266 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : secret "router-metrics-certs-default" not found
Sep 16 18:17:20.907298 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:20.906863    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:24.906857759 +0000 UTC m=+65.146646489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : configmap references non-existent config key: service-ca.crt
Sep 16 18:17:21.628284 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:21.628091    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-42dj7" event={"ID":"56254d14-1b92-4b27-aa73-678c951f78ef","Type":"ContainerStarted","Data":"0b5a495b0e37c70e8423253336e5f566224f04050c6745af45cb3c1ff17ccc1d"}
Sep 16 18:17:21.630006 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:21.629977    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh" event={"ID":"5eb43333-facd-4124-8053-8008cd52f251","Type":"ContainerStarted","Data":"de7a7540043bf26a4a2d269dcc67bdb0847d7d00593640a53739b42b7a31e7e3"}
Sep 16 18:17:21.631618 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:21.631576    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp" event={"ID":"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3","Type":"ContainerStarted","Data":"3fb91bf38fa58d3ab4201213e77674193a4a3ff8948a6124b6c2db4e3f2d3885"}
Sep 16 18:17:21.647688 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:21.647634    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-42dj7" podStartSLOduration=1.6926637740000001 podStartE2EDuration="4.647618185s" totalImagesPullingTime="2.954954411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:21.647076936 +0000 UTC m=+61.886865687" watchObservedRunningTime="2026-09-16 18:17:21.647618185 +0000 UTC m=+61.887406938"
Sep 16 18:17:21.688362 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:21.688317    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh" podStartSLOduration=1.723833084 podStartE2EDuration="4.688300599s" totalImagesPullingTime="2.964467515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:21.687584391 +0000 UTC m=+61.927373145" watchObservedRunningTime="2026-09-16 18:17:21.688300599 +0000 UTC m=+61.928089352"
Sep 16 18:17:21.688742 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:21.688714    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp" podStartSLOduration=1.705269176 podStartE2EDuration="4.688704579s" totalImagesPullingTime="2.983435403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:21.664698808 +0000 UTC m=+61.904487559" watchObservedRunningTime="2026-09-16 18:17:21.688704579 +0000 UTC m=+61.928493333"
Sep 16 18:17:22.132026 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.131991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:22.152023 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.151992    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s8brz"]
Sep 16 18:17:22.184103 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.184080    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s8brz"]
Sep 16 18:17:22.184248 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.184172    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.186864 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.186836    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 18:17:22.187003 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.186886    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 18:17:22.187003 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.186970    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9v5nj\""
Sep 16 18:17:22.216335 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.216306    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:22.217150 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.217126    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.217248 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.217167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/572ee141-d531-4716-b163-0f3c38d55648-data-volume\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.217248 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.217184    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.217364 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.217303    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/572ee141-d531-4716-b163-0f3c38d55648-crio-socket\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.217364 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.217342    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xlcb\" (UniqueName: \"kubernetes.io/projected/572ee141-d531-4716-b163-0f3c38d55648-kube-api-access-5xlcb\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.318474 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.318447    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/572ee141-d531-4716-b163-0f3c38d55648-data-volume\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.318572 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.318483    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.318647 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.318575    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/572ee141-d531-4716-b163-0f3c38d55648-crio-socket\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.318647 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:22.318609    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:22.318647 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.318626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xlcb\" (UniqueName: \"kubernetes.io/projected/572ee141-d531-4716-b163-0f3c38d55648-kube-api-access-5xlcb\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.318750 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:22.318669    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls podName:572ee141-d531-4716-b163-0f3c38d55648 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:22.818650489 +0000 UTC m=+63.058439222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s8brz" (UID: "572ee141-d531-4716-b163-0f3c38d55648") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:22.318750 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.318706    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/572ee141-d531-4716-b163-0f3c38d55648-crio-socket\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.318750 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.318721    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.318863 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:22.318797    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:17:22.318863 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:22.318850    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm podName:572ee141-d531-4716-b163-0f3c38d55648 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:22.818835252 +0000 UTC m=+63.058623987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s8brz" (UID: "572ee141-d531-4716-b163-0f3c38d55648") : configmap "kube-rbac-proxy" not found
Sep 16 18:17:22.318955 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.318858    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/572ee141-d531-4716-b163-0f3c38d55648-data-volume\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.329975 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.329955    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xlcb\" (UniqueName: \"kubernetes.io/projected/572ee141-d531-4716-b163-0f3c38d55648-kube-api-access-5xlcb\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.365197 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.365169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:22.563397 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.563364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:22.664445 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.664412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:22.822788 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.822718    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.822788 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.822778    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:22.822960 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:22.822830    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:17:22.822960 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:22.822882    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm podName:572ee141-d531-4716-b163-0f3c38d55648 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:23.82286626 +0000 UTC m=+64.062655010 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s8brz" (UID: "572ee141-d531-4716-b163-0f3c38d55648") : configmap "kube-rbac-proxy" not found
Sep 16 18:17:22.822960 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:22.822911    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:22.823077 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:22.822970    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls podName:572ee141-d531-4716-b163-0f3c38d55648 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:23.822957409 +0000 UTC m=+64.062746139 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s8brz" (UID: "572ee141-d531-4716-b163-0f3c38d55648") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:22.993737 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:22.993716    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:23.269773 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:23.269744    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:23.369237 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:23.369211    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:23.472878 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:23.472854    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:23.832110 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:23.832089    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:23.832227 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:23.832128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:23.832227 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:23.832211    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:17:23.832227 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:23.832220    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:23.832348 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:23.832264    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls podName:572ee141-d531-4716-b163-0f3c38d55648 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:25.832253616 +0000 UTC m=+66.072042346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s8brz" (UID: "572ee141-d531-4716-b163-0f3c38d55648") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:23.832348 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:23.832276    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm podName:572ee141-d531-4716-b163-0f3c38d55648 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:25.832270361 +0000 UTC m=+66.072059091 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s8brz" (UID: "572ee141-d531-4716-b163-0f3c38d55648") : configmap "kube-rbac-proxy" not found
Sep 16 18:17:24.055190 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.055174    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fxbpm_634753ab-bf55-4898-a91d-7b08037e8ac3/dns-node-resolver/0.log"
Sep 16 18:17:24.437087 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.437061    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:17:24.437370 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:24.437193    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:17:24.437370 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:24.437241    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls podName:bc29fcde-2de7-4474-beff-440b1e1724cf nodeName:}" failed. No retries permitted until 2026-09-16 18:17:56.437229434 +0000 UTC m=+96.677018164 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls") pod "dns-default-4dzgz" (UID: "bc29fcde-2de7-4474-beff-440b1e1724cf") : secret "dns-default-metrics-tls" not found
Sep 16 18:17:24.930232 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.930204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:24.940429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.940409    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:24.940498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.940442    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:17:24.940498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.940473    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjmc\" (UniqueName: \"kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc\") pod \"network-check-target-p8h5l\" (UID: \"9594a6fd-1697-4d79-a706-500845eb00f8\") " pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:17:24.940498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.940491    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:24.940643 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:24.940538    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:17:24.940643 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:24.940607    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:32.940571819 +0000 UTC m=+73.180360567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : secret "router-metrics-certs-default" not found
Sep 16 18:17:24.940643 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:24.940621    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:32.940615094 +0000 UTC m=+73.180403824 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : configmap references non-existent config key: service-ca.crt
Sep 16 18:17:24.943084 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.943066    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:17:24.943354 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.943341    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:17:24.953655 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.953610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b056af3-46bd-4340-8336-66d3e52a40f7-original-pull-secret\") pod \"global-pull-secret-syncer-2x4vg\" (UID: \"8b056af3-46bd-4340-8336-66d3e52a40f7\") " pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:17:24.954026 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.954013    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:17:24.963763 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:24.963745    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqjmc\" (UniqueName: \"kubernetes.io/projected/9594a6fd-1697-4d79-a706-500845eb00f8-kube-api-access-kqjmc\") pod \"network-check-target-p8h5l\" (UID: \"9594a6fd-1697-4d79-a706-500845eb00f8\") " pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:17:25.041058 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.041037    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:17:25.043675 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.043660    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:17:25.051808 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:25.051793    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 18:17:25.051855 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:25.051832    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs podName:c0001fd5-b780-42e5-a8f8-84f1f01076fe nodeName:}" failed. No retries permitted until 2026-09-16 18:18:29.051821353 +0000 UTC m=+129.291610083 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs") pod "network-metrics-daemon-mp8m7" (UID: "c0001fd5-b780-42e5-a8f8-84f1f01076fe") : secret "metrics-daemon-secret" not found
Sep 16 18:17:25.192504 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.192461    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2x4vg"
Sep 16 18:17:25.208222 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.208203    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bfgtt\""
Sep 16 18:17:25.216707 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.216691    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:17:25.257519 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.257476    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gzdnv_a094404a-9871-499f-bbf5-0223671705d4/node-ca/0.log"
Sep 16 18:17:25.396470 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.396440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:25.465002 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.464983    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2x4vg"]
Sep 16 18:17:25.466985 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:17:25.466962    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b056af3_46bd_4340_8336_66d3e52a40f7.slice/crio-635301ee4b8f4df65ee28405b54c2e61ba57107c5c5f7064aa317ef313de5605 WatchSource:0}: Error finding container 635301ee4b8f4df65ee28405b54c2e61ba57107c5c5f7064aa317ef313de5605: Status 404 returned error can't find the container with id 635301ee4b8f4df65ee28405b54c2e61ba57107c5c5f7064aa317ef313de5605
Sep 16 18:17:25.491575 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.491559    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p8h5l"]
Sep 16 18:17:25.493383 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:17:25.493362    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9594a6fd_1697_4d79_a706_500845eb00f8.slice/crio-25f67b8f63da9803747c112de2d0b8fc99bef4c379d470ca52872bedbbd92432 WatchSource:0}: Error finding container 25f67b8f63da9803747c112de2d0b8fc99bef4c379d470ca52872bedbbd92432: Status 404 returned error can't find the container with id 25f67b8f63da9803747c112de2d0b8fc99bef4c379d470ca52872bedbbd92432
Sep 16 18:17:25.499730 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.499704    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:25.642297 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.642272    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2x4vg" event={"ID":"8b056af3-46bd-4340-8336-66d3e52a40f7","Type":"ContainerStarted","Data":"635301ee4b8f4df65ee28405b54c2e61ba57107c5c5f7064aa317ef313de5605"}
Sep 16 18:17:25.643394 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.643375    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p8h5l" event={"ID":"9594a6fd-1697-4d79-a706-500845eb00f8","Type":"ContainerStarted","Data":"25f67b8f63da9803747c112de2d0b8fc99bef4c379d470ca52872bedbbd92432"}
Sep 16 18:17:25.657051 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.657032    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q4tb8_1ea3df26-130f-4aeb-a1e9-527f755670d5/serve-healthcheck-canary/0.log"
Sep 16 18:17:25.848077 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.848029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:25.848158 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.848096    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:25.848198 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:25.848159    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:17:25.848198 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:25.848170    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:25.848257 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:25.848204    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm podName:572ee141-d531-4716-b163-0f3c38d55648 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:29.848193109 +0000 UTC m=+70.087981839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s8brz" (UID: "572ee141-d531-4716-b163-0f3c38d55648") : configmap "kube-rbac-proxy" not found
Sep 16 18:17:25.848257 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:25.848216    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls podName:572ee141-d531-4716-b163-0f3c38d55648 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:29.848210773 +0000 UTC m=+70.087999503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s8brz" (UID: "572ee141-d531-4716-b163-0f3c38d55648") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:25.904256 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:25.904223    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:28.709880 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:28.709850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:29.462915 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:29.462879    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:29.560549 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:29.560516    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:29.672135 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:29.672105    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:29.880673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:29.880646    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:29.880989 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:29.880769    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:29.880989 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:29.880800    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:29.880989 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:29.880858    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:17:29.880989 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:29.880866    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls podName:572ee141-d531-4716-b163-0f3c38d55648 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:37.880845034 +0000 UTC m=+78.120633770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s8brz" (UID: "572ee141-d531-4716-b163-0f3c38d55648") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:29.880989 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:29.880909    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm podName:572ee141-d531-4716-b163-0f3c38d55648 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:37.880892874 +0000 UTC m=+78.120681624 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s8brz" (UID: "572ee141-d531-4716-b163-0f3c38d55648") : configmap "kube-rbac-proxy" not found
Sep 16 18:17:30.058051 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:30.058004    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:30.260694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:30.260659    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:30.361884 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:30.361854    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:30.560217 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:30.560160    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:30.658889 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:30.658865    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2x4vg" event={"ID":"8b056af3-46bd-4340-8336-66d3e52a40f7","Type":"ContainerStarted","Data":"6c3b3ba42c4359559192188f587c8040e9da7cee05cb8775d25c2722b638132c"}
Sep 16 18:17:30.660198 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:30.660178    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p8h5l" event={"ID":"9594a6fd-1697-4d79-a706-500845eb00f8","Type":"ContainerStarted","Data":"c7a7a28a617d66860de036885958db1a38b5a3911a3a406ea9b57050593ea774"}
Sep 16 18:17:30.660300 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:30.660290    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:17:30.680465 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:30.680419    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2x4vg" podStartSLOduration=66.305252435 podStartE2EDuration="1m10.680404685s" totalImagesPullingTime="4.37515225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:30.679943423 +0000 UTC m=+70.919732174" watchObservedRunningTime="2026-09-16 18:17:30.680404685 +0000 UTC m=+70.920193429"
Sep 16 18:17:30.698176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:30.698137    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p8h5l" podStartSLOduration=66.353683363 podStartE2EDuration="1m10.698125745s" totalImagesPullingTime="4.344442382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:30.6980734 +0000 UTC m=+70.937862152" watchObservedRunningTime="2026-09-16 18:17:30.698125745 +0000 UTC m=+70.937914494"
Sep 16 18:17:30.962181 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:30.962159    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:31.541887 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:31.541860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:31.620122 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:31.620095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:31.757772 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:31.757751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:31.955766 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:31.955738    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:32.556150 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:32.556120    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:32.751407 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:32.751388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:33.001878 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:33.001850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:33.001970 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:33.001907    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:33.002010 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:33.001994    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:49.00197952 +0000 UTC m=+89.241768250 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : configmap references non-existent config key: service-ca.crt
Sep 16 18:17:33.002010 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:33.002004    2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:17:33.002077 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:33.002038    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs podName:64086446-7f12-484e-b1e0-c0f57190258e nodeName:}" failed. No retries permitted until 2026-09-16 18:17:49.002027804 +0000 UTC m=+89.241816534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs") pod "router-default-7fcd5855b4-c2lsw" (UID: "64086446-7f12-484e-b1e0-c0f57190258e") : secret "router-metrics-certs-default" not found
Sep 16 18:17:33.166530 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:33.166501    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:33.289449 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:33.289396    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:33.382607 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:33.382568    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:33.949407 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:33.949387    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:34.159783 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:34.159755    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:34.222160 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:34.222116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:34.351566 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:34.351547    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:34.755933 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:34.755914    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:35.244071 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:35.244047    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:35.443061 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:35.443036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:35.656098 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:35.656046    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:35.756843 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:35.756820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:35.893203 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:35.893182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:35.991697 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:35.991659    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:36.167103 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:36.167077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:36.357747 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:36.357675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:36.959862 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:36.959835    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:37.496227 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:37.496203    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:37.669788 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:37.669764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:37.770062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:37.770005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:37.940910 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:37.940889    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:37.940999 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:37.940925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:37.941046 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:37.941033    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:37.941096 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:17:37.941087    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls podName:572ee141-d531-4716-b163-0f3c38d55648 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:53.94107163 +0000 UTC m=+94.180860377 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s8brz" (UID: "572ee141-d531-4716-b163-0f3c38d55648") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:17:37.941353 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:37.941337    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/572ee141-d531-4716-b163-0f3c38d55648-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:37.970068 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:37.970045    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:38.559506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:38.559477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:39.159797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:39.159762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:39.361286 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:39.361259    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:39.552728 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:39.552704    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:39.975789 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:39.975763    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:40.192410 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:40.192388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:44.722743 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.722616    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr"]
Sep 16 18:17:44.725907 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.725887    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr"
Sep 16 18:17:44.728498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.728479    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 18:17:44.728625 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.728606    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 18:17:44.729603 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.729568    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 18:17:44.729665 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.729627    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-8gbk7\""
Sep 16 18:17:44.730990 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.730973    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 16 18:17:44.737137 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.737118    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr"]
Sep 16 18:17:44.786951 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.786929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-689xw\" (UniqueName: \"kubernetes.io/projected/56a299c6-4d46-469b-9219-30246a1308d9-kube-api-access-689xw\") pod \"managed-serviceaccount-addon-agent-59d476b8fb-q2gfr\" (UID: \"56a299c6-4d46-469b-9219-30246a1308d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr"
Sep 16 18:17:44.787042 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.786964    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56a299c6-4d46-469b-9219-30246a1308d9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59d476b8fb-q2gfr\" (UID: \"56a299c6-4d46-469b-9219-30246a1308d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr"
Sep 16 18:17:44.855473 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.855450    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d65f57994-7qdks"]
Sep 16 18:17:44.858481 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.858466    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:44.861258 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.861232    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 18:17:44.861348 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.861323    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 18:17:44.861410 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.861384    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mlk5x\""
Sep 16 18:17:44.861635 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.861623    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 18:17:44.867726 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.867701    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 18:17:44.876139 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.876120    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d65f57994-7qdks"]
Sep 16 18:17:44.880245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.880223    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:44.887715 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.887696    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-689xw\" (UniqueName: \"kubernetes.io/projected/56a299c6-4d46-469b-9219-30246a1308d9-kube-api-access-689xw\") pod \"managed-serviceaccount-addon-agent-59d476b8fb-q2gfr\" (UID: \"56a299c6-4d46-469b-9219-30246a1308d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr"
Sep 16 18:17:44.887795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.887731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56a299c6-4d46-469b-9219-30246a1308d9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59d476b8fb-q2gfr\" (UID: \"56a299c6-4d46-469b-9219-30246a1308d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr"
Sep 16 18:17:44.890240 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.890215    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56a299c6-4d46-469b-9219-30246a1308d9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59d476b8fb-q2gfr\" (UID: \"56a299c6-4d46-469b-9219-30246a1308d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr"
Sep 16 18:17:44.896773 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.896756    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-689xw\" (UniqueName: \"kubernetes.io/projected/56a299c6-4d46-469b-9219-30246a1308d9-kube-api-access-689xw\") pod \"managed-serviceaccount-addon-agent-59d476b8fb-q2gfr\" (UID: \"56a299c6-4d46-469b-9219-30246a1308d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr"
Sep 16 18:17:44.978016 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.977948    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:44.988960 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.988942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cebccbde-4c2f-4e98-ac48-15aad4cc6317-trusted-ca\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:44.989024 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.988965    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cebccbde-4c2f-4e98-ac48-15aad4cc6317-registry-tls\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:44.989062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.989032    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cebccbde-4c2f-4e98-ac48-15aad4cc6317-installation-pull-secrets\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:44.989097 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.989066    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cebccbde-4c2f-4e98-ac48-15aad4cc6317-bound-sa-token\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:44.989129 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.989115    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cebccbde-4c2f-4e98-ac48-15aad4cc6317-image-registry-private-configuration\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:44.989161 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.989140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cebccbde-4c2f-4e98-ac48-15aad4cc6317-registry-certificates\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:44.989200 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.989159    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cebccbde-4c2f-4e98-ac48-15aad4cc6317-ca-trust-extracted\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:44.989240 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:44.989203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6mrl\" (UniqueName: \"kubernetes.io/projected/cebccbde-4c2f-4e98-ac48-15aad4cc6317-kube-api-access-g6mrl\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.049275 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.049248    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr"
Sep 16 18:17:45.089968 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.089945    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cebccbde-4c2f-4e98-ac48-15aad4cc6317-trusted-ca\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.090102 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.089983    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cebccbde-4c2f-4e98-ac48-15aad4cc6317-registry-tls\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.090347 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.090316    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cebccbde-4c2f-4e98-ac48-15aad4cc6317-installation-pull-secrets\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.090444 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.090390    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cebccbde-4c2f-4e98-ac48-15aad4cc6317-bound-sa-token\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.090514 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.090449    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cebccbde-4c2f-4e98-ac48-15aad4cc6317-image-registry-private-configuration\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.090514 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.090492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cebccbde-4c2f-4e98-ac48-15aad4cc6317-registry-certificates\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.090627 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.090522    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cebccbde-4c2f-4e98-ac48-15aad4cc6317-ca-trust-extracted\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.090892 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.090867    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cebccbde-4c2f-4e98-ac48-15aad4cc6317-ca-trust-extracted\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.091028 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.090932    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cebccbde-4c2f-4e98-ac48-15aad4cc6317-trusted-ca\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.091028 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.090959    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6mrl\" (UniqueName: \"kubernetes.io/projected/cebccbde-4c2f-4e98-ac48-15aad4cc6317-kube-api-access-g6mrl\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.091533 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.091450    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cebccbde-4c2f-4e98-ac48-15aad4cc6317-registry-certificates\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.092875 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.092838    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cebccbde-4c2f-4e98-ac48-15aad4cc6317-registry-tls\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.093612 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.093477    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cebccbde-4c2f-4e98-ac48-15aad4cc6317-installation-pull-secrets\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.093687 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.093628    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cebccbde-4c2f-4e98-ac48-15aad4cc6317-image-registry-private-configuration\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.100976 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.100467    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6mrl\" (UniqueName: \"kubernetes.io/projected/cebccbde-4c2f-4e98-ac48-15aad4cc6317-kube-api-access-g6mrl\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.101519 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.101498    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cebccbde-4c2f-4e98-ac48-15aad4cc6317-bound-sa-token\") pod \"image-registry-7d65f57994-7qdks\" (UID: \"cebccbde-4c2f-4e98-ac48-15aad4cc6317\") " pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.166438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.166413    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.331324 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.331270    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr"]
Sep 16 18:17:45.334683 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:17:45.334661    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56a299c6_4d46_469b_9219_30246a1308d9.slice/crio-c46fb9da0a0247cc7f417e55865aea4b72b63796846b9d93c3cd909aafa37a07 WatchSource:0}: Error finding container c46fb9da0a0247cc7f417e55865aea4b72b63796846b9d93c3cd909aafa37a07: Status 404 returned error can't find the container with id c46fb9da0a0247cc7f417e55865aea4b72b63796846b9d93c3cd909aafa37a07
Sep 16 18:17:45.441795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.441770    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d65f57994-7qdks"]
Sep 16 18:17:45.443979 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:17:45.443955    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcebccbde_4c2f_4e98_ac48_15aad4cc6317.slice/crio-372fec2282db2b3a4a5c1d65638e84c93bcc403b1b33563017307854e8328824 WatchSource:0}: Error finding container 372fec2282db2b3a4a5c1d65638e84c93bcc403b1b33563017307854e8328824: Status 404 returned error can't find the container with id 372fec2282db2b3a4a5c1d65638e84c93bcc403b1b33563017307854e8328824
Sep 16 18:17:45.704479 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.704443    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d65f57994-7qdks" event={"ID":"cebccbde-4c2f-4e98-ac48-15aad4cc6317","Type":"ContainerStarted","Data":"ce09db13cdd17f68cfe57eb9defbc808a4a049698a78ef4ec3deaa2e1219f5e0"}
Sep 16 18:17:45.704694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.704520    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d65f57994-7qdks" event={"ID":"cebccbde-4c2f-4e98-ac48-15aad4cc6317","Type":"ContainerStarted","Data":"372fec2282db2b3a4a5c1d65638e84c93bcc403b1b33563017307854e8328824"}
Sep 16 18:17:45.704694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.704545    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:17:45.705560 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.705540    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr" event={"ID":"56a299c6-4d46-469b-9219-30246a1308d9","Type":"ContainerStarted","Data":"c46fb9da0a0247cc7f417e55865aea4b72b63796846b9d93c3cd909aafa37a07"}
Sep 16 18:17:45.726033 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:45.725994    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d65f57994-7qdks" podStartSLOduration=1.725981048 podStartE2EDuration="1.725981048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:45.72596167 +0000 UTC m=+85.965750423" watchObservedRunningTime="2026-09-16 18:17:45.725981048 +0000 UTC m=+85.965769829"
Sep 16 18:17:47.714496 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:47.714470    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr" event={"ID":"56a299c6-4d46-469b-9219-30246a1308d9","Type":"ContainerStarted","Data":"cde5d4ba2e34176d592f127ebbd7b42924357c4183e4e45c2c3c56861d2c6c9b"}
Sep 16 18:17:47.733113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:47.733070    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59d476b8fb-q2gfr" podStartSLOduration=1.412463521 podStartE2EDuration="3.73305091s" totalImagesPullingTime="2.320587389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:47.732372335 +0000 UTC m=+87.972161087" watchObservedRunningTime="2026-09-16 18:17:47.73305091 +0000 UTC m=+87.972839662"
Sep 16 18:17:49.024416 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.024386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:49.024817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.024440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:49.024973 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.024955    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64086446-7f12-484e-b1e0-c0f57190258e-service-ca-bundle\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:49.026637 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.026619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64086446-7f12-484e-b1e0-c0f57190258e-metrics-certs\") pod \"router-default-7fcd5855b4-c2lsw\" (UID: \"64086446-7f12-484e-b1e0-c0f57190258e\") " pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:49.240628 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.240608    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-85f6b\""
Sep 16 18:17:49.248947 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.248933    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:49.477604 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.477567    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:49.526450 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.526429    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7fcd5855b4-c2lsw"]
Sep 16 18:17:49.530022 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:17:49.529991    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64086446_7f12_484e_b1e0_c0f57190258e.slice/crio-0e65d54d4c31ee2a9784de9b9755934374bf94b26b36168a71525d4087fa5140 WatchSource:0}: Error finding container 0e65d54d4c31ee2a9784de9b9755934374bf94b26b36168a71525d4087fa5140: Status 404 returned error can't find the container with id 0e65d54d4c31ee2a9784de9b9755934374bf94b26b36168a71525d4087fa5140
Sep 16 18:17:49.534613 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.534574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:49.721516 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.721480    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7fcd5855b4-c2lsw" event={"ID":"64086446-7f12-484e-b1e0-c0f57190258e","Type":"ContainerStarted","Data":"2b88f0618c3f0fac03b25143e15b034836b009e1caf3ac3fa3ba1dc11f49eedd"}
Sep 16 18:17:49.721657 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.721530    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7fcd5855b4-c2lsw" event={"ID":"64086446-7f12-484e-b1e0-c0f57190258e","Type":"ContainerStarted","Data":"0e65d54d4c31ee2a9784de9b9755934374bf94b26b36168a71525d4087fa5140"}
Sep 16 18:17:49.743543 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:49.743499    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7fcd5855b4-c2lsw" podStartSLOduration=32.7434852 podStartE2EDuration="32.7434852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:49.742213823 +0000 UTC m=+89.982002569" watchObservedRunningTime="2026-09-16 18:17:49.7434852 +0000 UTC m=+89.983273951"
Sep 16 18:17:50.249269 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:50.249240    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:50.251724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:50.251703    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:50.724905 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:50.724881    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:50.725956 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:50.725938    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7fcd5855b4-c2lsw"
Sep 16 18:17:51.637263 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:51.637235    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:51.749464 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:51.749439    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:51.849641 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:51.849619    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:52.212007 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:52.211982    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:53.958652 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:53.958624    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:53.960912 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:53.960893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/572ee141-d531-4716-b163-0f3c38d55648-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s8brz\" (UID: \"572ee141-d531-4716-b163-0f3c38d55648\") " pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:53.993868 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:53.993836    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s8brz"
Sep 16 18:17:54.262398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:54.262327    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s8brz"]
Sep 16 18:17:54.265650 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:17:54.265618    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod572ee141_d531_4716_b163_0f3c38d55648.slice/crio-5188e3a04511de1a9571ebe91b244619958f9eb8a569324b4df885fb5c588b7c WatchSource:0}: Error finding container 5188e3a04511de1a9571ebe91b244619958f9eb8a569324b4df885fb5c588b7c: Status 404 returned error can't find the container with id 5188e3a04511de1a9571ebe91b244619958f9eb8a569324b4df885fb5c588b7c
Sep 16 18:17:54.737299 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:54.737265    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s8brz" event={"ID":"572ee141-d531-4716-b163-0f3c38d55648","Type":"ContainerStarted","Data":"aedc6d51485d34010b5e73a1ed9155a9f4c3e58cef70db3cea2cbbe309226c8e"}
Sep 16 18:17:54.737514 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:54.737322    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s8brz" event={"ID":"572ee141-d531-4716-b163-0f3c38d55648","Type":"ContainerStarted","Data":"5188e3a04511de1a9571ebe91b244619958f9eb8a569324b4df885fb5c588b7c"}
Sep 16 18:17:55.741808 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:55.741774    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s8brz" event={"ID":"572ee141-d531-4716-b163-0f3c38d55648","Type":"ContainerStarted","Data":"1803d17165bcab38f8d5f31e2679be036fd1b604fce81dd989157a3cbd037c9c"}
Sep 16 18:17:55.896449 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:55.896416    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:55.973652 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:55.973620    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:56.333448 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:56.333412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:17:56.479927 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:56.479889    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:17:56.482363 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:56.482335    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc29fcde-2de7-4474-beff-440b1e1724cf-metrics-tls\") pod \"dns-default-4dzgz\" (UID: \"bc29fcde-2de7-4474-beff-440b1e1724cf\") " pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:17:56.501886 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:56.501860    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f8sdc\""
Sep 16 18:17:56.509960 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:56.509936    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:17:56.932121 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:56.932056    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4dzgz"]
Sep 16 18:17:56.934198 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:17:56.934176    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc29fcde_2de7_4474_beff_440b1e1724cf.slice/crio-60d302e409c8532087e7d24be3e6e4cc2e2e4b3bc370473e3aed6c4a5dd48801 WatchSource:0}: Error finding container 60d302e409c8532087e7d24be3e6e4cc2e2e4b3bc370473e3aed6c4a5dd48801: Status 404 returned error can't find the container with id 60d302e409c8532087e7d24be3e6e4cc2e2e4b3bc370473e3aed6c4a5dd48801
Sep 16 18:17:57.751027 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:57.750992    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s8brz" event={"ID":"572ee141-d531-4716-b163-0f3c38d55648","Type":"ContainerStarted","Data":"b27141faa7fc153812b5247e2ed9b86196555e15ac841fef3a201bbee71b74a3"}
Sep 16 18:17:57.752323 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:57.752296    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4dzgz" event={"ID":"bc29fcde-2de7-4474-beff-440b1e1724cf","Type":"ContainerStarted","Data":"60d302e409c8532087e7d24be3e6e4cc2e2e4b3bc370473e3aed6c4a5dd48801"}
Sep 16 18:17:57.772765 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:57.772723    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s8brz" podStartSLOduration=33.426084075 podStartE2EDuration="35.772704816s" totalImagesPullingTime="2.346620741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:17:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:57.771508444 +0000 UTC m=+98.011297197" watchObservedRunningTime="2026-09-16 18:17:57.772704816 +0000 UTC m=+98.012493565"
Sep 16 18:17:58.757141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:58.757111    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4dzgz" event={"ID":"bc29fcde-2de7-4474-beff-440b1e1724cf","Type":"ContainerStarted","Data":"c32895497af347484d000981f1c60fd9d6f337c9dd8052dc789b1fe999a6a62a"}
Sep 16 18:17:58.757496 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:58.757160    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4dzgz" event={"ID":"bc29fcde-2de7-4474-beff-440b1e1724cf","Type":"ContainerStarted","Data":"7f3590fe221ba1d70b20bdab7f1c6def3cafb45fd98c5cbe0ed782e349fe2f16"}
Sep 16 18:17:58.778674 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:58.778633    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4dzgz" podStartSLOduration=65.324373176 podStartE2EDuration="1m6.778621137s" totalImagesPullingTime="1.454247961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:16:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:58.77815293 +0000 UTC m=+99.017941682" watchObservedRunningTime="2026-09-16 18:17:58.778621137 +0000 UTC m=+99.018409890"
Sep 16 18:17:59.760224 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:17:59.760202    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:18:00.124292 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:00.124229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:01.666286 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:01.666256    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p8h5l"
Sep 16 18:18:02.043274 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.043200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:02.288584 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.288560    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"]
Sep 16 18:18:02.292824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.292809    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.295223 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.295178    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 16 18:18:02.295310 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.295279    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 16 18:18:02.296304 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.296285    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qfs9t\""
Sep 16 18:18:02.296398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.296309    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 18:18:02.296398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.296345    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 18:18:02.296494 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.296406    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 18:18:02.306173 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.306148    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"]
Sep 16 18:18:02.309695 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.309666    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"]
Sep 16 18:18:02.312767 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.312750    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.315839 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.315825    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 16 18:18:02.315923 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.315860    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 16 18:18:02.316113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.316100    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 16 18:18:02.316325 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.316310    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cdjqp\""
Sep 16 18:18:02.331868 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.331843    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"]
Sep 16 18:18:02.333291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.332765    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4p87f"]
Sep 16 18:18:02.336707 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.336690    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.339174 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.339153    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 18:18:02.339266 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.339240    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 18:18:02.339328 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.339267    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-975dg\""
Sep 16 18:18:02.339328 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.339317    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 18:18:02.363655 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.363623    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:02.425146 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425125    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-metrics-client-ca\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.425237 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425149    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-sys\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.425237 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425166    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-accelerators-collector-config\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.425237 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425182    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f26f53fe-ffed-4063-bac9-b569e5fb12dc-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.425237 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/23bc5849-687e-4a9d-9eab-10899639bb62-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.425434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425251    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgzjr\" (UniqueName: \"kubernetes.io/projected/23bc5849-687e-4a9d-9eab-10899639bb62-kube-api-access-tgzjr\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.425434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425270    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfxb6\" (UniqueName: \"kubernetes.io/projected/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-kube-api-access-kfxb6\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.425434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425341    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/23bc5849-687e-4a9d-9eab-10899639bb62-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.425434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425380    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f26f53fe-ffed-4063-bac9-b569e5fb12dc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.425434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425408    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f26f53fe-ffed-4063-bac9-b569e5fb12dc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.425700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425447    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-wtmp\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.425700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425501    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.425700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425529    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23bc5849-687e-4a9d-9eab-10899639bb62-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.425700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425578    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-root\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.425700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425624    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-tls\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.425700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425651    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-textfile\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.425700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23bc5849-687e-4a9d-9eab-10899639bb62-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.425929 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425721    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23bc5849-687e-4a9d-9eab-10899639bb62-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.425929 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.425748    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njhcc\" (UniqueName: \"kubernetes.io/projected/f26f53fe-ffed-4063-bac9-b569e5fb12dc-kube-api-access-njhcc\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.428326 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.428297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:02.526028 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526002    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-metrics-client-ca\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.526137 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526038    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-sys\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.526137 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-accelerators-collector-config\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.526137 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526077    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f26f53fe-ffed-4063-bac9-b569e5fb12dc-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.526137 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526096    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/23bc5849-687e-4a9d-9eab-10899639bb62-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.526137 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526115    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgzjr\" (UniqueName: \"kubernetes.io/projected/23bc5849-687e-4a9d-9eab-10899639bb62-kube-api-access-tgzjr\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.526388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfxb6\" (UniqueName: \"kubernetes.io/projected/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-kube-api-access-kfxb6\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.526388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/23bc5849-687e-4a9d-9eab-10899639bb62-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.526388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f26f53fe-ffed-4063-bac9-b569e5fb12dc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.526388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526213    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f26f53fe-ffed-4063-bac9-b569e5fb12dc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.526388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-wtmp\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.526388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526278    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.526388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526307    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23bc5849-687e-4a9d-9eab-10899639bb62-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.526388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-sys\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.526388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-root\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.526824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526410    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-tls\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.526824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-textfile\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.526824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23bc5849-687e-4a9d-9eab-10899639bb62-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.526824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526507    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23bc5849-687e-4a9d-9eab-10899639bb62-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.526824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526540    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njhcc\" (UniqueName: \"kubernetes.io/projected/f26f53fe-ffed-4063-bac9-b569e5fb12dc-kube-api-access-njhcc\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.526824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.526646    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-metrics-client-ca\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.527100 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:18:02.527000    2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 16 18:18:02.527100 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:18:02.527086    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f26f53fe-ffed-4063-bac9-b569e5fb12dc-openshift-state-metrics-tls podName:f26f53fe-ffed-4063-bac9-b569e5fb12dc nodeName:}" failed. No retries permitted until 2026-09-16 18:18:03.027067428 +0000 UTC m=+103.266856168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f26f53fe-ffed-4063-bac9-b569e5fb12dc-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-7lf6t" (UID: "f26f53fe-ffed-4063-bac9-b569e5fb12dc") : secret "openshift-state-metrics-tls" not found
Sep 16 18:18:02.527219 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.527132    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-root\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.527219 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.527168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f26f53fe-ffed-4063-bac9-b569e5fb12dc-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.527347 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.527327    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23bc5849-687e-4a9d-9eab-10899639bb62-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.527411 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.527387    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:02.527411 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.527395    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/23bc5849-687e-4a9d-9eab-10899639bb62-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.527511 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:18:02.527482    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 16 18:18:02.527563 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:18:02.527534    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-tls podName:82ea4326-263a-4fd2-bd4b-3c76f0b874fb nodeName:}" failed. No retries permitted until 2026-09-16 18:18:03.027518095 +0000 UTC m=+103.267306833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-tls") pod "node-exporter-4p87f" (UID: "82ea4326-263a-4fd2-bd4b-3c76f0b874fb") : secret "node-exporter-tls" not found
Sep 16 18:18:02.527692 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.527635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-wtmp\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.527811 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.527715    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-accelerators-collector-config\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.527869 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.527850    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/23bc5849-687e-4a9d-9eab-10899639bb62-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.528145 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.528122    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-textfile\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.529627 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.529605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.529733 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.529719    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/23bc5849-687e-4a9d-9eab-10899639bb62-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.530427 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.530408    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f26f53fe-ffed-4063-bac9-b569e5fb12dc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.530427 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.530419    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23bc5849-687e-4a9d-9eab-10899639bb62-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.535857 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.535836    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfxb6\" (UniqueName: \"kubernetes.io/projected/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-kube-api-access-kfxb6\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:02.535942 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.535839    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njhcc\" (UniqueName: \"kubernetes.io/projected/f26f53fe-ffed-4063-bac9-b569e5fb12dc-kube-api-access-njhcc\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:02.535942 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.535892    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgzjr\" (UniqueName: \"kubernetes.io/projected/23bc5849-687e-4a9d-9eab-10899639bb62-kube-api-access-tgzjr\") pod \"kube-state-metrics-59fbdd77d-9lhhj\" (UID: \"23bc5849-687e-4a9d-9eab-10899639bb62\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.620742 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.620693    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"
Sep 16 18:18:02.626383 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.626361    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:02.726224 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.726174    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:02.827791 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.827768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:02.894517 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.894459    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj"]
Sep 16 18:18:02.897097 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:18:02.897072    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23bc5849_687e_4a9d_9eab_10899639bb62.slice/crio-a5bd9cd99854b1adc3e46415f284687f41f0a7f077c5cc46d0a8a06e7d8f751b WatchSource:0}: Error finding container a5bd9cd99854b1adc3e46415f284687f41f0a7f077c5cc46d0a8a06e7d8f751b: Status 404 returned error can't find the container with id a5bd9cd99854b1adc3e46415f284687f41f0a7f077c5cc46d0a8a06e7d8f751b
Sep 16 18:18:02.898241 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:02.898212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:03.030330 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.030306    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f26f53fe-ffed-4063-bac9-b569e5fb12dc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:03.030414 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.030348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-tls\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:03.032627 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.032611    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/82ea4326-263a-4fd2-bd4b-3c76f0b874fb-node-exporter-tls\") pod \"node-exporter-4p87f\" (UID: \"82ea4326-263a-4fd2-bd4b-3c76f0b874fb\") " pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:03.032678 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.032639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f26f53fe-ffed-4063-bac9-b569e5fb12dc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7lf6t\" (UID: \"f26f53fe-ffed-4063-bac9-b569e5fb12dc\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:03.201022 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.200999    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"
Sep 16 18:18:03.244233 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.244204    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4p87f"
Sep 16 18:18:03.256686 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:18:03.256654    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ea4326_263a_4fd2_bd4b_3c76f0b874fb.slice/crio-6755ea17862b5456cd61d8d1b72221b3cd0164be54710524c82f0a16c333d7d5 WatchSource:0}: Error finding container 6755ea17862b5456cd61d8d1b72221b3cd0164be54710524c82f0a16c333d7d5: Status 404 returned error can't find the container with id 6755ea17862b5456cd61d8d1b72221b3cd0164be54710524c82f0a16c333d7d5
Sep 16 18:18:03.283038 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.283011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:03.377835 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.377786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:03.470649 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.470513    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t"]
Sep 16 18:18:03.472925 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:18:03.472906    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26f53fe_ffed_4063_bac9_b569e5fb12dc.slice/crio-97902f09427f75216bf45e6eda159b9ac43a34835078585a77c894763cbb3b56 WatchSource:0}: Error finding container 97902f09427f75216bf45e6eda159b9ac43a34835078585a77c894763cbb3b56: Status 404 returned error can't find the container with id 97902f09427f75216bf45e6eda159b9ac43a34835078585a77c894763cbb3b56
Sep 16 18:18:03.663873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.663845    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:03.778354 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.778263    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj" event={"ID":"23bc5849-687e-4a9d-9eab-10899639bb62","Type":"ContainerStarted","Data":"a5bd9cd99854b1adc3e46415f284687f41f0a7f077c5cc46d0a8a06e7d8f751b"}
Sep 16 18:18:03.780050 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.779991    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4p87f" event={"ID":"82ea4326-263a-4fd2-bd4b-3c76f0b874fb","Type":"ContainerStarted","Data":"6755ea17862b5456cd61d8d1b72221b3cd0164be54710524c82f0a16c333d7d5"}
Sep 16 18:18:03.782026 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.781995    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t" event={"ID":"f26f53fe-ffed-4063-bac9-b569e5fb12dc","Type":"ContainerStarted","Data":"955e907c3f102851b8a6d3a0d6d27540a4958895503bd321ff9790e9b85214ff"}
Sep 16 18:18:03.782152 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.782044    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t" event={"ID":"f26f53fe-ffed-4063-bac9-b569e5fb12dc","Type":"ContainerStarted","Data":"e86d478d4f08f69efb1fbe4557b303b1a1424dd6fd48adf01d50c02174059ffa"}
Sep 16 18:18:03.782152 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:03.782071    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t" event={"ID":"f26f53fe-ffed-4063-bac9-b569e5fb12dc","Type":"ContainerStarted","Data":"97902f09427f75216bf45e6eda159b9ac43a34835078585a77c894763cbb3b56"}
Sep 16 18:18:04.381950 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.381924    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-668478c48-cz8dl"]
Sep 16 18:18:04.412468 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.412443    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-668478c48-cz8dl"]
Sep 16 18:18:04.412610 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.412549    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.415685 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.415663    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 16 18:18:04.415789 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.415713    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 16 18:18:04.416225 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.416186    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 16 18:18:04.416320 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.416213    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 16 18:18:04.416431 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.416417    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 16 18:18:04.416525 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.416414    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f8fr2pccn0335\""
Sep 16 18:18:04.416759 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.416746    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q9zh9\""
Sep 16 18:18:04.543490 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.543432    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-grpc-tls\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.543676 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.543517    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s69zt\" (UniqueName: \"kubernetes.io/projected/9f867486-44dc-4c83-aca6-1ce2e12091b8-kube-api-access-s69zt\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.543676 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.543562    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.543676 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.543625    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f867486-44dc-4c83-aca6-1ce2e12091b8-metrics-client-ca\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.543676 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.543671    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-tls\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.543905 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.543706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.543905 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.543793    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.543905 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.543831    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.644392 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.644323    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-grpc-tls\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.644392 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.644361    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s69zt\" (UniqueName: \"kubernetes.io/projected/9f867486-44dc-4c83-aca6-1ce2e12091b8-kube-api-access-s69zt\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.644392 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.644379    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.644640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.644397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f867486-44dc-4c83-aca6-1ce2e12091b8-metrics-client-ca\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.644640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.644441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-tls\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.644640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.644469    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.644640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.644499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.644640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.644524    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.645313 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.645285    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f867486-44dc-4c83-aca6-1ce2e12091b8-metrics-client-ca\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.647018 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.646992    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.647214 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.647194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-grpc-tls\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.647540 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.647518    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.647683 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.647664    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.647731 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.647704    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-tls\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.647768 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.647725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f867486-44dc-4c83-aca6-1ce2e12091b8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.652262 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.652246    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s69zt\" (UniqueName: \"kubernetes.io/projected/9f867486-44dc-4c83-aca6-1ce2e12091b8-kube-api-access-s69zt\") pod \"thanos-querier-668478c48-cz8dl\" (UID: \"9f867486-44dc-4c83-aca6-1ce2e12091b8\") " pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:04.721288 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:04.721263    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:05.170809 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.170774    2562 patch_prober.go:28] interesting pod/image-registry-7d65f57994-7qdks container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 18:18:05.171180 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.170830    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7d65f57994-7qdks" podUID="cebccbde-4c2f-4e98-ac48-15aad4cc6317" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:18:05.289328 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.289296    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:05.354341 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.354307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:05.419175 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.419149    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:05.512031 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.512003    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:05.705682 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.705626    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-668478c48-cz8dl"]
Sep 16 18:18:05.710662 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:18:05.710636    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f867486_44dc_4c83_aca6_1ce2e12091b8.slice/crio-f7a61d1b26be98282952f40cdb371e3412f5086f88570b9b5f14237c80acb154 WatchSource:0}: Error finding container f7a61d1b26be98282952f40cdb371e3412f5086f88570b9b5f14237c80acb154: Status 404 returned error can't find the container with id f7a61d1b26be98282952f40cdb371e3412f5086f88570b9b5f14237c80acb154
Sep 16 18:18:05.792529 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.792499    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t" event={"ID":"f26f53fe-ffed-4063-bac9-b569e5fb12dc","Type":"ContainerStarted","Data":"d553cddc962616c34592dfeff5a84685baf29ea2f4a917628067a65361dd295e"}
Sep 16 18:18:05.795414 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.795388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj" event={"ID":"23bc5849-687e-4a9d-9eab-10899639bb62","Type":"ContainerStarted","Data":"182490fceab4f3522bfb8a8cfe0bcf4231da83e9454e32b9eb0ee4ad6d8243d0"}
Sep 16 18:18:05.795533 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.795441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj" event={"ID":"23bc5849-687e-4a9d-9eab-10899639bb62","Type":"ContainerStarted","Data":"83cf42d430aff78e376245a679994cb831ea5dd030ed017bae9877998de0a2de"}
Sep 16 18:18:05.795533 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.795469    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj" event={"ID":"23bc5849-687e-4a9d-9eab-10899639bb62","Type":"ContainerStarted","Data":"eb07889a6ac40634ae04857667d7f3865d9436c3a72522c0da97b1d3e073f8f4"}
Sep 16 18:18:05.797085 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.797055    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl" event={"ID":"9f867486-44dc-4c83-aca6-1ce2e12091b8","Type":"ContainerStarted","Data":"f7a61d1b26be98282952f40cdb371e3412f5086f88570b9b5f14237c80acb154"}
Sep 16 18:18:05.799076 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.799053    2562 generic.go:356] "Generic (PLEG): container finished" podID="82ea4326-263a-4fd2-bd4b-3c76f0b874fb" containerID="8235205ef820536e3dddb80b57d9fb626cdde15cf18aaafc962d78c17466c77e" exitCode=0
Sep 16 18:18:05.799187 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.799100    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4p87f" event={"ID":"82ea4326-263a-4fd2-bd4b-3c76f0b874fb","Type":"ContainerDied","Data":"8235205ef820536e3dddb80b57d9fb626cdde15cf18aaafc962d78c17466c77e"}
Sep 16 18:18:05.813051 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.813004    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7lf6t" podStartSLOduration=1.972921975 podStartE2EDuration="3.812986541s" totalImagesPullingTime="1.840064566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:18:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:18:05.812914607 +0000 UTC m=+106.052703360" watchObservedRunningTime="2026-09-16 18:18:05.812986541 +0000 UTC m=+106.052775295"
Sep 16 18:18:05.832585 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:05.832531    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9lhhj" podStartSLOduration=1.298613238 podStartE2EDuration="3.832512738s" totalImagesPullingTime="2.5338995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:18:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:18:05.832336936 +0000 UTC m=+106.072125686" watchObservedRunningTime="2026-09-16 18:18:05.832512738 +0000 UTC m=+106.072301492"
Sep 16 18:18:06.714826 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:06.714786    2562 patch_prober.go:28] interesting pod/image-registry-7d65f57994-7qdks container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 18:18:06.715269 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:06.714847    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7d65f57994-7qdks" podUID="cebccbde-4c2f-4e98-ac48-15aad4cc6317" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:18:06.806230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:06.806196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4p87f" event={"ID":"82ea4326-263a-4fd2-bd4b-3c76f0b874fb","Type":"ContainerStarted","Data":"9baf7c298ecd2654d0618b203f2590c188cc0ac32d1e536eab9fe30a9d37693d"}
Sep 16 18:18:06.806384 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:06.806290    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4p87f" event={"ID":"82ea4326-263a-4fd2-bd4b-3c76f0b874fb","Type":"ContainerStarted","Data":"14b3a4fd1f7ae904e4584e125d011172e88c11cf9bd61eeaa6cac9fae97ac942"}
Sep 16 18:18:06.833605 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:06.833549    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4p87f" podStartSLOduration=2.569291731 podStartE2EDuration="4.833532423s" totalImagesPullingTime="2.17399862s" totalInitContainerRuntime="90.242072ms" isStatefulPod=true podCreationTimestamp="2026-09-16 18:18:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:18:06.832269859 +0000 UTC m=+107.072058612" watchObservedRunningTime="2026-09-16 18:18:06.833532423 +0000 UTC m=+107.073321174"
Sep 16 18:18:07.059346 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:07.059260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:07.469847 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:07.469809    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:07.810790 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:07.810762    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl" event={"ID":"9f867486-44dc-4c83-aca6-1ce2e12091b8","Type":"ContainerStarted","Data":"b7a3527252f6c81774f91ac1fe407fe627ceb02eb72c895c7966b117e8699081"}
Sep 16 18:18:08.528852 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:08.528818    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:08.816620 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:08.816576    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl" event={"ID":"9f867486-44dc-4c83-aca6-1ce2e12091b8","Type":"ContainerStarted","Data":"659ca72d6e27ceb1ae182399140281d2b5a2e1bf5b9c44622332167d3f666ca9"}
Sep 16 18:18:08.816947 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:08.816655    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl" event={"ID":"9f867486-44dc-4c83-aca6-1ce2e12091b8","Type":"ContainerStarted","Data":"d9c7121f94c22bdcc7f825703bf9ff3f5452a07ec61bca25ee5db4f2cd2ae541"}
Sep 16 18:18:09.766663 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:09.766636    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4dzgz"
Sep 16 18:18:09.824157 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:09.824128    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl" event={"ID":"9f867486-44dc-4c83-aca6-1ce2e12091b8","Type":"ContainerStarted","Data":"635b2e6f0ad07f734a88e7abe72f535f62b39b00fe39090805e1a3db427000ff"}
Sep 16 18:18:09.824497 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:09.824190    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl" event={"ID":"9f867486-44dc-4c83-aca6-1ce2e12091b8","Type":"ContainerStarted","Data":"72d75b7f245ad2e39ea5ad518658145e1f2fab76580f4e6445ac929fab8f129e"}
Sep 16 18:18:09.824497 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:09.824216    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl" event={"ID":"9f867486-44dc-4c83-aca6-1ce2e12091b8","Type":"ContainerStarted","Data":"ccfe905ed43dc1ab6b9a4c5be8ccf04782d3efa242a8297671c852ff81a5bb99"}
Sep 16 18:18:09.824497 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:09.824271    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:09.850139 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:09.850095    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl" podStartSLOduration=2.988681846 podStartE2EDuration="5.85008003s" totalImagesPullingTime="2.861398184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:18:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:18:09.848473704 +0000 UTC m=+110.088262456" watchObservedRunningTime="2026-09-16 18:18:09.85008003 +0000 UTC m=+110.089868781"
Sep 16 18:18:14.437694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.437662    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-zvj95"]
Sep 16 18:18:14.440833 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.440804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:14.441084 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.441071    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-zvj95"
Sep 16 18:18:14.443747 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.443728    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 18:18:14.443890 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.443872    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4mtmn\""
Sep 16 18:18:14.443942 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.443898    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 18:18:14.454127 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.454107    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-zvj95"]
Sep 16 18:18:14.506480 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.506461    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:14.523857 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.523829    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbhzc\" (UniqueName: \"kubernetes.io/projected/24c6189d-c787-49c4-9a04-1617da5ad6de-kube-api-access-lbhzc\") pod \"downloads-76f99584f4-zvj95\" (UID: \"24c6189d-c787-49c4-9a04-1617da5ad6de\") " pod="openshift-console/downloads-76f99584f4-zvj95"
Sep 16 18:18:14.572454 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.572430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:14.624218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.624198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbhzc\" (UniqueName: \"kubernetes.io/projected/24c6189d-c787-49c4-9a04-1617da5ad6de-kube-api-access-lbhzc\") pod \"downloads-76f99584f4-zvj95\" (UID: \"24c6189d-c787-49c4-9a04-1617da5ad6de\") " pod="openshift-console/downloads-76f99584f4-zvj95"
Sep 16 18:18:14.632543 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.632518    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbhzc\" (UniqueName: \"kubernetes.io/projected/24c6189d-c787-49c4-9a04-1617da5ad6de-kube-api-access-lbhzc\") pod \"downloads-76f99584f4-zvj95\" (UID: \"24c6189d-c787-49c4-9a04-1617da5ad6de\") " pod="openshift-console/downloads-76f99584f4-zvj95"
Sep 16 18:18:14.718459 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.718402    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:14.749551 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.749529    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-zvj95"
Sep 16 18:18:14.888215 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.888190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:14.951778 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:14.951750    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:15.025337 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:15.025289    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-zvj95"]
Sep 16 18:18:15.029292 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:18:15.029267    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c6189d_c787_49c4_9a04_1617da5ad6de.slice/crio-84d1441870a89ece6514400c874199aff84f1dcfd6c687b7521c36254ba564c9 WatchSource:0}: Error finding container 84d1441870a89ece6514400c874199aff84f1dcfd6c687b7521c36254ba564c9: Status 404 returned error can't find the container with id 84d1441870a89ece6514400c874199aff84f1dcfd6c687b7521c36254ba564c9
Sep 16 18:18:15.115127 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:15.115101    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:15.170713 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:15.170691    2562 patch_prober.go:28] interesting pod/image-registry-7d65f57994-7qdks container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 18:18:15.170793 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:15.170727    2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7d65f57994-7qdks" podUID="cebccbde-4c2f-4e98-ac48-15aad4cc6317" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:18:15.319948 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:15.319889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:15.411354 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:15.411330    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:15.528891 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:15.528869    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:15.833696 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:15.833672    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-668478c48-cz8dl"
Sep 16 18:18:15.847156 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:15.847127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-zvj95" event={"ID":"24c6189d-c787-49c4-9a04-1617da5ad6de","Type":"ContainerStarted","Data":"84d1441870a89ece6514400c874199aff84f1dcfd6c687b7521c36254ba564c9"}
Sep 16 18:18:16.305715 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:16.305682    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:16.509681 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:16.509643    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:16.621784 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:16.621703    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:16.715077 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:16.715053    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d65f57994-7qdks"
Sep 16 18:18:16.863242 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:16.863209    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:16.971500 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:16.971467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:17.546078 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:17.546041    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:17.647328 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:17.647285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:17.814927 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:17.814849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:17.913543 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:17.913500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:18.596675 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:18.596638    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:18.655073 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:18.655035    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:20.268698 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:20.268661    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:20.328007 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:20.327967    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:23.723488 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:23.723439    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:25.241859 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.241831    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-688c89db57-mmb6p"]
Sep 16 18:18:25.245648 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.245625    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.249006 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.248982    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 18:18:25.249774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.249740    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jldnv\""
Sep 16 18:18:25.249774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.249746    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 18:18:25.250076 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.250001    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 18:18:25.250076 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.250021    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 18:18:25.250260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.250241    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 18:18:25.256550 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.256529    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688c89db57-mmb6p"]
Sep 16 18:18:25.318455 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.318424    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-config\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.318606 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.318474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-service-ca\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.318606 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.318536    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-serving-cert\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.318606 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.318555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-oauth-config\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.318606 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.318578    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh5lp\" (UniqueName: \"kubernetes.io/projected/f5959da3-39ce-4b5f-a8a3-0bce123142f0-kube-api-access-lh5lp\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.318778 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.318641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-oauth-serving-cert\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.419026 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.418988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-service-ca\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.419192 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.419051    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-serving-cert\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.419192 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.419079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-oauth-config\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.419192 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.419101    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh5lp\" (UniqueName: \"kubernetes.io/projected/f5959da3-39ce-4b5f-a8a3-0bce123142f0-kube-api-access-lh5lp\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.419192 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.419140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-oauth-serving-cert\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.419192 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.419174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-config\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.419993 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.419841    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-service-ca\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.419993 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.419857    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-oauth-serving-cert\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.419993 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.419977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-config\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.421987 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.421963    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-serving-cert\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.422106 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.422081    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-oauth-config\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.433913 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.433891    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh5lp\" (UniqueName: \"kubernetes.io/projected/f5959da3-39ce-4b5f-a8a3-0bce123142f0-kube-api-access-lh5lp\") pod \"console-688c89db57-mmb6p\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") " pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:25.558289 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:25.558197    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:26.123346 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:26.123314    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:26.223853 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:26.223820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:26.556055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:26.556025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:26.656391 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:26.656357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:27.651434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:27.651394    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:27.752271 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:27.752238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:28.714150 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:28.714084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:28.788407 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:28.788374    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:29.149963 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:29.149885    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:18:29.152679 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:29.152656    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0001fd5-b780-42e5-a8f8-84f1f01076fe-metrics-certs\") pod \"network-metrics-daemon-mp8m7\" (UID: \"c0001fd5-b780-42e5-a8f8-84f1f01076fe\") " pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:18:29.264461 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:29.264429    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:29.371865 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:29.371831    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:29.401970 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:29.401902    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5rgdl\""
Sep 16 18:18:29.410447 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:29.410422    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp8m7"
Sep 16 18:18:29.452358 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:29.452332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:30.733897 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:30.733860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:30.792628 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:30.792541    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:31.071342 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:31.071006    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mp8m7"]
Sep 16 18:18:31.072860 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:31.072803    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688c89db57-mmb6p"]
Sep 16 18:18:31.074479 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:18:31.074445    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5959da3_39ce_4b5f_a8a3_0bce123142f0.slice/crio-eabbeb65b37fa2ed541d83762c3f29fc6e0e34ce3e03730f3c42c4cfa3819003 WatchSource:0}: Error finding container eabbeb65b37fa2ed541d83762c3f29fc6e0e34ce3e03730f3c42c4cfa3819003: Status 404 returned error can't find the container with id eabbeb65b37fa2ed541d83762c3f29fc6e0e34ce3e03730f3c42c4cfa3819003
Sep 16 18:18:31.075101 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:18:31.075075    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0001fd5_b780_42e5_a8f8_84f1f01076fe.slice/crio-5f58c5d8f638b20c1cd50f4ff27dc9da790ee179782c0142b05e3a266888b01e WatchSource:0}: Error finding container 5f58c5d8f638b20c1cd50f4ff27dc9da790ee179782c0142b05e3a266888b01e: Status 404 returned error can't find the container with id 5f58c5d8f638b20c1cd50f4ff27dc9da790ee179782c0142b05e3a266888b01e
Sep 16 18:18:31.916661 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:31.916617    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mp8m7" event={"ID":"c0001fd5-b780-42e5-a8f8-84f1f01076fe","Type":"ContainerStarted","Data":"5f58c5d8f638b20c1cd50f4ff27dc9da790ee179782c0142b05e3a266888b01e"}
Sep 16 18:18:31.920699 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:31.920575    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-zvj95" event={"ID":"24c6189d-c787-49c4-9a04-1617da5ad6de","Type":"ContainerStarted","Data":"a35c78e4c4df789adfbd056beeb684c72e4129cd891816dd9de1e2c1fd6cc984"}
Sep 16 18:18:31.923480 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:31.923374    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-zvj95"
Sep 16 18:18:31.926083 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:31.926056    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688c89db57-mmb6p" event={"ID":"f5959da3-39ce-4b5f-a8a3-0bce123142f0","Type":"ContainerStarted","Data":"eabbeb65b37fa2ed541d83762c3f29fc6e0e34ce3e03730f3c42c4cfa3819003"}
Sep 16 18:18:31.935758 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:31.935396    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-zvj95"
Sep 16 18:18:31.944799 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:31.944738    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-zvj95" podStartSLOduration=2.084541991 podStartE2EDuration="17.944718448s" totalImagesPullingTime="15.860176457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:18:31.942054983 +0000 UTC m=+132.181843735" watchObservedRunningTime="2026-09-16 18:18:31.944718448 +0000 UTC m=+132.184507201"
Sep 16 18:18:32.653720 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:32.653691    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:32.757784 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:32.757752    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:32.936454 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:32.936102    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mp8m7" event={"ID":"c0001fd5-b780-42e5-a8f8-84f1f01076fe","Type":"ContainerStarted","Data":"cc15af1eb328c7f32da5a1d4fb6c8fb2606792f734334421118caef2ef3fded7"}
Sep 16 18:18:33.943697 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:33.943659    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mp8m7" event={"ID":"c0001fd5-b780-42e5-a8f8-84f1f01076fe","Type":"ContainerStarted","Data":"578895c70a0ef612c7d00e8bd649dff4778c92116ecdbfdbc12881d5f8e993c6"}
Sep 16 18:18:33.964046 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:33.963986    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mp8m7" podStartSLOduration=132.427020977 podStartE2EDuration="2m13.963964039s" totalImagesPullingTime="1.536943062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:18:33.963074076 +0000 UTC m=+134.202862850" watchObservedRunningTime="2026-09-16 18:18:33.963964039 +0000 UTC m=+134.203752807"
Sep 16 18:18:34.079328 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.079291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:34.140302 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.140216    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:34.545399 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.545365    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-648d4cdb9c-lxm4w"]
Sep 16 18:18:34.562972 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.562947    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-648d4cdb9c-lxm4w"]
Sep 16 18:18:34.563115 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.563061    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.577714 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.576782    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 18:18:34.595964 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.595939    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-service-ca\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.596096 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.595976    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-oauth-serving-cert\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.596096 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.596007    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx6jw\" (UniqueName: \"kubernetes.io/projected/62cc9adc-501e-46ee-8eaf-d77d14648325-kube-api-access-nx6jw\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.596096 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.596085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-trusted-ca-bundle\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.596245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.596152    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-console-config\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.596245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.596197    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-serving-cert\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.596245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.596216    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-oauth-config\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.672986 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.672954    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:34.696949 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.696916    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-console-config\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.697076 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.696972    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-serving-cert\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.697076 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.696999    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-oauth-config\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.697076 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.697065    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-service-ca\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.697233 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.697090    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-oauth-serving-cert\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.697233 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.697116    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx6jw\" (UniqueName: \"kubernetes.io/projected/62cc9adc-501e-46ee-8eaf-d77d14648325-kube-api-access-nx6jw\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.697233 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.697163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-trusted-ca-bundle\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.697662 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.697641    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-console-config\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.698004 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.697980    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-oauth-serving-cert\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.698100 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.698003    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-trusted-ca-bundle\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.698241 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.698223    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-service-ca\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.699875 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.699853    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-serving-cert\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.700491 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.700468    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-oauth-config\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.706508 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.706474    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx6jw\" (UniqueName: \"kubernetes.io/projected/62cc9adc-501e-46ee-8eaf-d77d14648325-kube-api-access-nx6jw\") pod \"console-648d4cdb9c-lxm4w\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") " pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.773554 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.773521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:34.880428 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.880356    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:34.924236 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.924205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:34.998843 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:34.998769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:35.271640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:35.271611    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-648d4cdb9c-lxm4w"]
Sep 16 18:18:35.275268 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:18:35.275240    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62cc9adc_501e_46ee_8eaf_d77d14648325.slice/crio-673d75c8c26c402abeb5121ca542cdd2d73c3bba197c14a6dfee9b07c75bcf69 WatchSource:0}: Error finding container 673d75c8c26c402abeb5121ca542cdd2d73c3bba197c14a6dfee9b07c75bcf69: Status 404 returned error can't find the container with id 673d75c8c26c402abeb5121ca542cdd2d73c3bba197c14a6dfee9b07c75bcf69
Sep 16 18:18:35.953835 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:35.953792    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688c89db57-mmb6p" event={"ID":"f5959da3-39ce-4b5f-a8a3-0bce123142f0","Type":"ContainerStarted","Data":"4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6"}
Sep 16 18:18:35.955702 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:35.955661    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648d4cdb9c-lxm4w" event={"ID":"62cc9adc-501e-46ee-8eaf-d77d14648325","Type":"ContainerStarted","Data":"3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293"}
Sep 16 18:18:35.955842 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:35.955710    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648d4cdb9c-lxm4w" event={"ID":"62cc9adc-501e-46ee-8eaf-d77d14648325","Type":"ContainerStarted","Data":"673d75c8c26c402abeb5121ca542cdd2d73c3bba197c14a6dfee9b07c75bcf69"}
Sep 16 18:18:35.976233 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:35.976190    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-688c89db57-mmb6p" podStartSLOduration=7.055090807 podStartE2EDuration="10.976175518s" totalImagesPullingTime="3.921084711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:18:35.973864966 +0000 UTC m=+136.213653720" watchObservedRunningTime="2026-09-16 18:18:35.976175518 +0000 UTC m=+136.215964270"
Sep 16 18:18:36.962218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:36.962180    2562 generic.go:356] "Generic (PLEG): container finished" podID="5eb43333-facd-4124-8053-8008cd52f251" containerID="de7a7540043bf26a4a2d269dcc67bdb0847d7d00593640a53739b42b7a31e7e3" exitCode=0
Sep 16 18:18:36.962660 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:36.962250    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh" event={"ID":"5eb43333-facd-4124-8053-8008cd52f251","Type":"ContainerDied","Data":"de7a7540043bf26a4a2d269dcc67bdb0847d7d00593640a53739b42b7a31e7e3"}
Sep 16 18:18:36.962660 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:36.962643    2562 scope.go:117] "RemoveContainer" containerID="de7a7540043bf26a4a2d269dcc67bdb0847d7d00593640a53739b42b7a31e7e3"
Sep 16 18:18:36.979744 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:36.979698    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-648d4cdb9c-lxm4w" podStartSLOduration=2.979681369 podStartE2EDuration="2.979681369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:18:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:18:35.994367281 +0000 UTC m=+136.234156034" watchObservedRunningTime="2026-09-16 18:18:36.979681369 +0000 UTC m=+137.219470121"
Sep 16 18:18:37.968828 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:37.968795    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xw4vh" event={"ID":"5eb43333-facd-4124-8053-8008cd52f251","Type":"ContainerStarted","Data":"00f56a91638e962f3de9f5bc91cf2f935597c9943c424a3f9d7a0fa9e4fad3f8"}
Sep 16 18:18:44.881314 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:44.881280    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:44.881752 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:44.881337    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:44.885478 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:44.885450    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:44.999700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:44.999678    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:18:45.063170 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:45.063143    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688c89db57-mmb6p"]
Sep 16 18:18:45.558569 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:45.558543    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:18:46.741907 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:46.741872    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:46.821669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:46.821639    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:47.331279 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:47.331251    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:47.428718 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:47.428691    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:48.333514 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:48.333478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:48.405871 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:48.405844    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:50.887154 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:50.887120    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:50.953335 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:50.953303    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:18:52.022744 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:52.022684    2562 generic.go:356] "Generic (PLEG): container finished" podID="8e3a2033-5283-49f6-8407-ad0a2fe0a1d3" containerID="3fb91bf38fa58d3ab4201213e77674193a4a3ff8948a6124b6c2db4e3f2d3885" exitCode=0
Sep 16 18:18:52.023082 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:52.022763    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp" event={"ID":"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3","Type":"ContainerDied","Data":"3fb91bf38fa58d3ab4201213e77674193a4a3ff8948a6124b6c2db4e3f2d3885"}
Sep 16 18:18:52.023082 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:52.023041    2562 scope.go:117] "RemoveContainer" containerID="3fb91bf38fa58d3ab4201213e77674193a4a3ff8948a6124b6c2db4e3f2d3885"
Sep 16 18:18:52.024433 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:52.024416    2562 generic.go:356] "Generic (PLEG): container finished" podID="56254d14-1b92-4b27-aa73-678c951f78ef" containerID="0b5a495b0e37c70e8423253336e5f566224f04050c6745af45cb3c1ff17ccc1d" exitCode=0
Sep 16 18:18:52.024524 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:52.024460    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-42dj7" event={"ID":"56254d14-1b92-4b27-aa73-678c951f78ef","Type":"ContainerDied","Data":"0b5a495b0e37c70e8423253336e5f566224f04050c6745af45cb3c1ff17ccc1d"}
Sep 16 18:18:52.024719 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:52.024707    2562 scope.go:117] "RemoveContainer" containerID="0b5a495b0e37c70e8423253336e5f566224f04050c6745af45cb3c1ff17ccc1d"
Sep 16 18:18:53.030001 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:53.029963    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-42dj7" event={"ID":"56254d14-1b92-4b27-aa73-678c951f78ef","Type":"ContainerStarted","Data":"f1899f2addf4160fc0249dd1797598e90209be096a369c40adffb37e7f9efbc8"}
Sep 16 18:18:53.031562 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:18:53.031541    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-hh4gp" event={"ID":"8e3a2033-5283-49f6-8407-ad0a2fe0a1d3","Type":"ContainerStarted","Data":"44130c25098ff69db4a70f29edbb034582812b9d3fa2d3981a029d8054d8e06f"}
Sep 16 18:19:05.900362 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:05.900316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:05.970628 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:05.970578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:10.083055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.082999    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-688c89db57-mmb6p" podUID="f5959da3-39ce-4b5f-a8a3-0bce123142f0" containerName="console" containerID="cri-o://4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6" gracePeriod=15
Sep 16 18:19:10.355642 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.355566    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:19:10.463757 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.463729    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-config\") pod \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") "
Sep 16 18:19:10.463874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.463761    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-oauth-config\") pod \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") "
Sep 16 18:19:10.463874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.463787    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh5lp\" (UniqueName: \"kubernetes.io/projected/f5959da3-39ce-4b5f-a8a3-0bce123142f0-kube-api-access-lh5lp\") pod \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") "
Sep 16 18:19:10.463961 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.463915    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-serving-cert\") pod \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") "
Sep 16 18:19:10.464002 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.463986    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-oauth-serving-cert\") pod \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") "
Sep 16 18:19:10.464050 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.464027    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-service-ca\") pod \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\" (UID: \"f5959da3-39ce-4b5f-a8a3-0bce123142f0\") "
Sep 16 18:19:10.464113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.464085    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-config" (OuterVolumeSpecName: "console-config") pod "f5959da3-39ce-4b5f-a8a3-0bce123142f0" (UID: "f5959da3-39ce-4b5f-a8a3-0bce123142f0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:19:10.464293 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.464270    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:19:10.464293 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.464269    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f5959da3-39ce-4b5f-a8a3-0bce123142f0" (UID: "f5959da3-39ce-4b5f-a8a3-0bce123142f0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:19:10.464440 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.464358    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-service-ca" (OuterVolumeSpecName: "service-ca") pod "f5959da3-39ce-4b5f-a8a3-0bce123142f0" (UID: "f5959da3-39ce-4b5f-a8a3-0bce123142f0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:19:10.466178 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.466156    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f5959da3-39ce-4b5f-a8a3-0bce123142f0" (UID: "f5959da3-39ce-4b5f-a8a3-0bce123142f0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:19:10.466370 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.466355    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5959da3-39ce-4b5f-a8a3-0bce123142f0-kube-api-access-lh5lp" (OuterVolumeSpecName: "kube-api-access-lh5lp") pod "f5959da3-39ce-4b5f-a8a3-0bce123142f0" (UID: "f5959da3-39ce-4b5f-a8a3-0bce123142f0"). InnerVolumeSpecName "kube-api-access-lh5lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:19:10.466414 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.466363    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f5959da3-39ce-4b5f-a8a3-0bce123142f0" (UID: "f5959da3-39ce-4b5f-a8a3-0bce123142f0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:19:10.564760 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.564738    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:19:10.564847 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.564765    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-oauth-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:19:10.564847 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.564780    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5959da3-39ce-4b5f-a8a3-0bce123142f0-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:19:10.564847 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.564793    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5959da3-39ce-4b5f-a8a3-0bce123142f0-console-oauth-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:19:10.564847 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:10.564806    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh5lp\" (UniqueName: \"kubernetes.io/projected/f5959da3-39ce-4b5f-a8a3-0bce123142f0-kube-api-access-lh5lp\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:19:11.106010 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:11.105981    2562 generic.go:356] "Generic (PLEG): container finished" podID="f5959da3-39ce-4b5f-a8a3-0bce123142f0" containerID="4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6" exitCode=0
Sep 16 18:19:11.106372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:11.106022    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688c89db57-mmb6p"
Sep 16 18:19:11.106372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:11.106048    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688c89db57-mmb6p" event={"ID":"f5959da3-39ce-4b5f-a8a3-0bce123142f0","Type":"ContainerDied","Data":"4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6"}
Sep 16 18:19:11.106372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:11.106085    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688c89db57-mmb6p" event={"ID":"f5959da3-39ce-4b5f-a8a3-0bce123142f0","Type":"ContainerDied","Data":"eabbeb65b37fa2ed541d83762c3f29fc6e0e34ce3e03730f3c42c4cfa3819003"}
Sep 16 18:19:11.106372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:11.106104    2562 scope.go:117] "RemoveContainer" containerID="4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6"
Sep 16 18:19:11.114479 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:11.114462    2562 scope.go:117] "RemoveContainer" containerID="4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6"
Sep 16 18:19:11.114740 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:19:11.114722    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6\": container with ID starting with 4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6 not found: ID does not exist" containerID="4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6"
Sep 16 18:19:11.114791 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:11.114748    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6"} err="failed to get container status \"4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6\": rpc error: code = NotFound desc = could not find container \"4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6\": container with ID starting with 4621dafaf97d2a74a79d4a187f979b0f34bc7375331e112ff10c216618e7f4d6 not found: ID does not exist"
Sep 16 18:19:11.127009 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:11.126987    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688c89db57-mmb6p"]
Sep 16 18:19:11.131207 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:11.131186    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-688c89db57-mmb6p"]
Sep 16 18:19:12.283960 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:12.283894    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5959da3-39ce-4b5f-a8a3-0bce123142f0" path="/var/lib/kubelet/pods/f5959da3-39ce-4b5f-a8a3-0bce123142f0/volumes"
Sep 16 18:19:21.576884 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:21.576842    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:21.825992 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:21.825959    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:26.671770 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.671742    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"]
Sep 16 18:19:26.672114 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.672007    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5959da3-39ce-4b5f-a8a3-0bce123142f0" containerName="console"
Sep 16 18:19:26.672114 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.672017    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5959da3-39ce-4b5f-a8a3-0bce123142f0" containerName="console"
Sep 16 18:19:26.672114 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.672070    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5959da3-39ce-4b5f-a8a3-0bce123142f0" containerName="console"
Sep 16 18:19:26.674661 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.674642    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.677148 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.677127    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 16 18:19:26.677229 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.677200    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 16 18:19:26.677287 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.677260    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 16 18:19:26.677396 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.677321    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 16 18:19:26.677396 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.677355    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bj5s8\""
Sep 16 18:19:26.677951 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.677933    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 16 18:19:26.684199 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.684181    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 16 18:19:26.692846 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.692826    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"]
Sep 16 18:19:26.775676 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.775654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-serving-certs-ca-bundle\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.775794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.775688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-federate-client-tls\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.775794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.775724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-metrics-client-ca\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.775794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.775761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbw6v\" (UniqueName: \"kubernetes.io/projected/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-kube-api-access-cbw6v\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.775794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.775778    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.775943 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.775848    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-secret-telemeter-client\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.775943 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.775877    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.775943 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.775896    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-telemeter-client-tls\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.876329 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.876304    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.876413 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.876332    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-telemeter-client-tls\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.876413 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.876354    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-serving-certs-ca-bundle\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.876413 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.876375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-federate-client-tls\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.876413 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.876401    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-metrics-client-ca\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.876619 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.876422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbw6v\" (UniqueName: \"kubernetes.io/projected/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-kube-api-access-cbw6v\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.876619 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.876445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.876619 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.876479    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-secret-telemeter-client\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.877315 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.877289    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-serving-certs-ca-bundle\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.877409 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.877314    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-metrics-client-ca\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.877474 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.877420    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.878927 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.878905    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-federate-client-tls\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.879027 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.879007    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-telemeter-client-tls\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.879459 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.879431    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.879530 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.879489    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-secret-telemeter-client\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.885639 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.885617    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbw6v\" (UniqueName: \"kubernetes.io/projected/7f591335-8f7c-41d5-b9a1-4ab18335b7b9-kube-api-access-cbw6v\") pod \"telemeter-client-b8cfbb97b-twrv5\" (UID: \"7f591335-8f7c-41d5-b9a1-4ab18335b7b9\") " pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:26.983684 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:26.983623    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"
Sep 16 18:19:27.261405 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:27.261324    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b8cfbb97b-twrv5"]
Sep 16 18:19:27.264833 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:19:27.264806    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f591335_8f7c_41d5_b9a1_4ab18335b7b9.slice/crio-c6bcba1a0bfccb02624c64f5265fd0d52587b2fae2f8e9adf3eb97b6e2fc27df WatchSource:0}: Error finding container c6bcba1a0bfccb02624c64f5265fd0d52587b2fae2f8e9adf3eb97b6e2fc27df: Status 404 returned error can't find the container with id c6bcba1a0bfccb02624c64f5265fd0d52587b2fae2f8e9adf3eb97b6e2fc27df
Sep 16 18:19:28.170015 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:28.169980    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5" event={"ID":"7f591335-8f7c-41d5-b9a1-4ab18335b7b9","Type":"ContainerStarted","Data":"c6bcba1a0bfccb02624c64f5265fd0d52587b2fae2f8e9adf3eb97b6e2fc27df"}
Sep 16 18:19:30.179831 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:30.179797    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5" event={"ID":"7f591335-8f7c-41d5-b9a1-4ab18335b7b9","Type":"ContainerStarted","Data":"b4dc6c5c6c3714f66cd014b9c0acdef4ee42e30bc24e65a8516dd24d40b8fe15"}
Sep 16 18:19:31.185942 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.185908    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5" event={"ID":"7f591335-8f7c-41d5-b9a1-4ab18335b7b9","Type":"ContainerStarted","Data":"f3142a0f5fa59889bc67de4bc520ac86d570c5ecc87a274583fdfe435b9779db"}
Sep 16 18:19:31.186277 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.185991    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5" event={"ID":"7f591335-8f7c-41d5-b9a1-4ab18335b7b9","Type":"ContainerStarted","Data":"f5c0eec754d98bc949d57b4574fe044d6858c106f2051a35c17bb672c464cf41"}
Sep 16 18:19:31.222986 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.222939    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-b8cfbb97b-twrv5" podStartSLOduration=1.990571026 podStartE2EDuration="5.222923242s" totalImagesPullingTime="3.232352216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:19:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:19:31.210083018 +0000 UTC m=+191.449871769" watchObservedRunningTime="2026-09-16 18:19:31.222923242 +0000 UTC m=+191.462712000"
Sep 16 18:19:31.300635 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.300611    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-97c47c456-kjbzp"]
Sep 16 18:19:31.302735 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.302716    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.315362 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.315341    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-97c47c456-kjbzp"]
Sep 16 18:19:31.360667 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.360635    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:31.408784 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.408760    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-service-ca\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.408882 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.408799    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-trusted-ca-bundle\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.408882 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.408836    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hvlc\" (UniqueName: \"kubernetes.io/projected/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-kube-api-access-7hvlc\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.409000 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.408928    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-serving-cert\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.409000 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.408961    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-config\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.409087 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.409010    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-oauth-config\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.409087 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.409064    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-oauth-serving-cert\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.423013 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.422985    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:31.509734 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.509677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hvlc\" (UniqueName: \"kubernetes.io/projected/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-kube-api-access-7hvlc\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.509734 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.509716    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-serving-cert\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.509874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.509736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-config\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.509874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.509824    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-oauth-config\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.509874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.509865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-oauth-serving-cert\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.510020 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.509928    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-service-ca\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.510020 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.509954    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-trusted-ca-bundle\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.510545 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.510435    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-config\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.510671 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.510624    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-service-ca\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.510671 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.510624    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-oauth-serving-cert\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.510791 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.510764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:31.511249 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.511228    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-trusted-ca-bundle\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.512393 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.512373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-serving-cert\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.512703 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.512687    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-oauth-config\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.517984 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.517965    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hvlc\" (UniqueName: \"kubernetes.io/projected/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-kube-api-access-7hvlc\") pod \"console-97c47c456-kjbzp\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") " pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.611298 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.611279    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:31.880366 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:31.880344    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-97c47c456-kjbzp"]
Sep 16 18:19:31.883123 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:19:31.883094    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52a4bd7_0ea0_4954_aac4_82c1eb6ec71a.slice/crio-3bab09f72ab01fb4a28db1105dc7cef080f0afcac432ab45d523f6ce086db991 WatchSource:0}: Error finding container 3bab09f72ab01fb4a28db1105dc7cef080f0afcac432ab45d523f6ce086db991: Status 404 returned error can't find the container with id 3bab09f72ab01fb4a28db1105dc7cef080f0afcac432ab45d523f6ce086db991
Sep 16 18:19:32.147920 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:32.147841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:32.191320 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:32.191288    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-97c47c456-kjbzp" event={"ID":"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a","Type":"ContainerStarted","Data":"56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437"}
Sep 16 18:19:32.191736 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:32.191331    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-97c47c456-kjbzp" event={"ID":"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a","Type":"ContainerStarted","Data":"3bab09f72ab01fb4a28db1105dc7cef080f0afcac432ab45d523f6ce086db991"}
Sep 16 18:19:32.212374 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:32.212330    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-97c47c456-kjbzp" podStartSLOduration=1.212316702 podStartE2EDuration="1.212316702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:19:32.210368855 +0000 UTC m=+192.450157609" watchObservedRunningTime="2026-09-16 18:19:32.212316702 +0000 UTC m=+192.452105454"
Sep 16 18:19:32.256291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:32.256269    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:34.431412 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:34.431377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:41.611835 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:41.611803    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:41.612202 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:41.611883    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:41.616323 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:41.616302    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:42.233507 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:42.233488    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:19:42.290929 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:42.290901    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-648d4cdb9c-lxm4w"]
Sep 16 18:19:42.402935 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:42.402904    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:42.896086 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:42.896054    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:42.968528 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:42.968469    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:19:49.744041 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:19:49.744008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:07.310300 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.310250    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-648d4cdb9c-lxm4w" podUID="62cc9adc-501e-46ee-8eaf-d77d14648325" containerName="console" containerID="cri-o://3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293" gracePeriod=15
Sep 16 18:20:07.547423 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.547401    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:20:07.650917 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.650850    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx6jw\" (UniqueName: \"kubernetes.io/projected/62cc9adc-501e-46ee-8eaf-d77d14648325-kube-api-access-nx6jw\") pod \"62cc9adc-501e-46ee-8eaf-d77d14648325\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") "
Sep 16 18:20:07.651035 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.650937    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-oauth-config\") pod \"62cc9adc-501e-46ee-8eaf-d77d14648325\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") "
Sep 16 18:20:07.651035 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.650971    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-trusted-ca-bundle\") pod \"62cc9adc-501e-46ee-8eaf-d77d14648325\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") "
Sep 16 18:20:07.651035 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.651010    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-serving-cert\") pod \"62cc9adc-501e-46ee-8eaf-d77d14648325\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") "
Sep 16 18:20:07.651229 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.651034    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-service-ca\") pod \"62cc9adc-501e-46ee-8eaf-d77d14648325\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") "
Sep 16 18:20:07.651229 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.651063    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-oauth-serving-cert\") pod \"62cc9adc-501e-46ee-8eaf-d77d14648325\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") "
Sep 16 18:20:07.651229 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.651092    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-console-config\") pod \"62cc9adc-501e-46ee-8eaf-d77d14648325\" (UID: \"62cc9adc-501e-46ee-8eaf-d77d14648325\") "
Sep 16 18:20:07.651441 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.651412    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "62cc9adc-501e-46ee-8eaf-d77d14648325" (UID: "62cc9adc-501e-46ee-8eaf-d77d14648325"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:20:07.651510 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.651426    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-service-ca" (OuterVolumeSpecName: "service-ca") pod "62cc9adc-501e-46ee-8eaf-d77d14648325" (UID: "62cc9adc-501e-46ee-8eaf-d77d14648325"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:20:07.651663 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.651643    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-console-config" (OuterVolumeSpecName: "console-config") pod "62cc9adc-501e-46ee-8eaf-d77d14648325" (UID: "62cc9adc-501e-46ee-8eaf-d77d14648325"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:20:07.651786 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.651453    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "62cc9adc-501e-46ee-8eaf-d77d14648325" (UID: "62cc9adc-501e-46ee-8eaf-d77d14648325"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:20:07.652984 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.652960    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62cc9adc-501e-46ee-8eaf-d77d14648325-kube-api-access-nx6jw" (OuterVolumeSpecName: "kube-api-access-nx6jw") pod "62cc9adc-501e-46ee-8eaf-d77d14648325" (UID: "62cc9adc-501e-46ee-8eaf-d77d14648325"). InnerVolumeSpecName "kube-api-access-nx6jw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:20:07.653080 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.653055    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "62cc9adc-501e-46ee-8eaf-d77d14648325" (UID: "62cc9adc-501e-46ee-8eaf-d77d14648325"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:20:07.653151 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.653129    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "62cc9adc-501e-46ee-8eaf-d77d14648325" (UID: "62cc9adc-501e-46ee-8eaf-d77d14648325"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:20:07.751979 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.751958    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx6jw\" (UniqueName: \"kubernetes.io/projected/62cc9adc-501e-46ee-8eaf-d77d14648325-kube-api-access-nx6jw\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:20:07.751979 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.751977    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-oauth-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:20:07.752091 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.751986    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-trusted-ca-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:20:07.752091 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.751994    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62cc9adc-501e-46ee-8eaf-d77d14648325-console-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:20:07.752091 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.752002    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:20:07.752091 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.752010    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-oauth-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:20:07.752091 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:07.752018    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62cc9adc-501e-46ee-8eaf-d77d14648325-console-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:20:08.328566 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:08.328543    2562 generic.go:356] "Generic (PLEG): container finished" podID="62cc9adc-501e-46ee-8eaf-d77d14648325" containerID="3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293" exitCode=0
Sep 16 18:20:08.328948 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:08.328607    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648d4cdb9c-lxm4w"
Sep 16 18:20:08.328948 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:08.328627    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648d4cdb9c-lxm4w" event={"ID":"62cc9adc-501e-46ee-8eaf-d77d14648325","Type":"ContainerDied","Data":"3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293"}
Sep 16 18:20:08.328948 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:08.328669    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648d4cdb9c-lxm4w" event={"ID":"62cc9adc-501e-46ee-8eaf-d77d14648325","Type":"ContainerDied","Data":"673d75c8c26c402abeb5121ca542cdd2d73c3bba197c14a6dfee9b07c75bcf69"}
Sep 16 18:20:08.328948 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:08.328688    2562 scope.go:117] "RemoveContainer" containerID="3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293"
Sep 16 18:20:08.336640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:08.336620    2562 scope.go:117] "RemoveContainer" containerID="3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293"
Sep 16 18:20:08.336897 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:20:08.336880    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293\": container with ID starting with 3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293 not found: ID does not exist" containerID="3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293"
Sep 16 18:20:08.336952 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:08.336909    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293"} err="failed to get container status \"3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293\": rpc error: code = NotFound desc = could not find container \"3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293\": container with ID starting with 3d58ac4abcc432926ad1285107fbdb33e89ad3aae3f35d1b1a02ac072cf2c293 not found: ID does not exist"
Sep 16 18:20:08.344697 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:08.344677    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-648d4cdb9c-lxm4w"]
Sep 16 18:20:08.350214 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:08.350195    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-648d4cdb9c-lxm4w"]
Sep 16 18:20:10.284663 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:10.284632    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62cc9adc-501e-46ee-8eaf-d77d14648325" path="/var/lib/kubelet/pods/62cc9adc-501e-46ee-8eaf-d77d14648325/volumes"
Sep 16 18:20:22.986257 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:22.986215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:32.402826 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:32.402792    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:32.502967 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:32.502940    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:36.902285 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:36.902239    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:36.968067 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:36.968031    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:37.075263 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.075240    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d5db6bb8-g8k2p"]
Sep 16 18:20:37.075555 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.075541    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62cc9adc-501e-46ee-8eaf-d77d14648325" containerName="console"
Sep 16 18:20:37.075623 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.075558    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="62cc9adc-501e-46ee-8eaf-d77d14648325" containerName="console"
Sep 16 18:20:37.075665 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.075639    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="62cc9adc-501e-46ee-8eaf-d77d14648325" containerName="console"
Sep 16 18:20:37.079790 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.079767    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.092300 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.092278    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d5db6bb8-g8k2p"]
Sep 16 18:20:37.130769 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.130742    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:37.150710 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.150684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-service-ca\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.150805 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.150736    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-serving-cert\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.150805 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.150770    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-oauth-config\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.150805 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.150792    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-oauth-serving-cert\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.150959 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.150824    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-config\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.150959 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.150873    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-trusted-ca-bundle\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.150959 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.150904    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmqbg\" (UniqueName: \"kubernetes.io/projected/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-kube-api-access-vmqbg\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.192707 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.192684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:37.251831 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.251804    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-service-ca\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.251923 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.251854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-serving-cert\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.251923 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.251886    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-oauth-config\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.251923 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.251909    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-oauth-serving-cert\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.252075 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.251940    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-config\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.252121 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.252079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-trusted-ca-bundle\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.252179 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.252126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmqbg\" (UniqueName: \"kubernetes.io/projected/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-kube-api-access-vmqbg\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.252515 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.252489    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-service-ca\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.252630 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.252564    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-oauth-serving-cert\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.252685 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.252631    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-config\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.252939 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.252920    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-trusted-ca-bundle\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.254439 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.254412    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-serving-cert\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.254439 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.254428    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-oauth-config\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.260720 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.260703    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmqbg\" (UniqueName: \"kubernetes.io/projected/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-kube-api-access-vmqbg\") pod \"console-7d5db6bb8-g8k2p\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") " pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.388176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.388151    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:37.658467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:37.658352    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d5db6bb8-g8k2p"]
Sep 16 18:20:37.661094 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:20:37.661069    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c2b462_4cc1_4a13_85a5_2b536f0e04ba.slice/crio-236e4b6569645c46fa0e2a709f20d51e848722122da9276440dbf9a9afe56d46 WatchSource:0}: Error finding container 236e4b6569645c46fa0e2a709f20d51e848722122da9276440dbf9a9afe56d46: Status 404 returned error can't find the container with id 236e4b6569645c46fa0e2a709f20d51e848722122da9276440dbf9a9afe56d46
Sep 16 18:20:38.440074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:38.440044    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5db6bb8-g8k2p" event={"ID":"82c2b462-4cc1-4a13-85a5-2b536f0e04ba","Type":"ContainerStarted","Data":"90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5"}
Sep 16 18:20:38.440412 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:38.440087    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5db6bb8-g8k2p" event={"ID":"82c2b462-4cc1-4a13-85a5-2b536f0e04ba","Type":"ContainerStarted","Data":"236e4b6569645c46fa0e2a709f20d51e848722122da9276440dbf9a9afe56d46"}
Sep 16 18:20:38.459909 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:38.459861    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d5db6bb8-g8k2p" podStartSLOduration=1.459847491 podStartE2EDuration="1.459847491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:20:38.458686204 +0000 UTC m=+258.698474955" watchObservedRunningTime="2026-09-16 18:20:38.459847491 +0000 UTC m=+258.699636242"
Sep 16 18:20:47.388963 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:47.388882    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:47.388963 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:47.388955    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:47.393172 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:47.393151    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:47.476736 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:47.476715    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:20:47.530856 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:47.530817    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-97c47c456-kjbzp"]
Sep 16 18:20:47.639468 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:47.639386    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:47.739627 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:47.739582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:47.841034 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:47.841011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:48.944749 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:48.944713    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:20:49.110623 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:20:49.110584    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:21:05.891936 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:05.891900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:21:05.992261 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:05.992231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:21:12.556383 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.556326    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-97c47c456-kjbzp" podUID="a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" containerName="console" containerID="cri-o://56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437" gracePeriod=15
Sep 16 18:21:12.794367 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.794341    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:21:12.894172 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.894117    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-oauth-serving-cert\") pod \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") "
Sep 16 18:21:12.894172 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.894147    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-service-ca\") pod \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") "
Sep 16 18:21:12.894353 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.894188    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hvlc\" (UniqueName: \"kubernetes.io/projected/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-kube-api-access-7hvlc\") pod \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") "
Sep 16 18:21:12.894353 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.894211    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-serving-cert\") pod \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") "
Sep 16 18:21:12.894353 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.894232    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-trusted-ca-bundle\") pod \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") "
Sep 16 18:21:12.894353 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.894246    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-oauth-config\") pod \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") "
Sep 16 18:21:12.894353 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.894280    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-config\") pod \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\" (UID: \"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a\") "
Sep 16 18:21:12.894629 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.894499    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-service-ca" (OuterVolumeSpecName: "service-ca") pod "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" (UID: "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:21:12.894629 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.894549    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" (UID: "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:21:12.894781 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.894761    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-config" (OuterVolumeSpecName: "console-config") pod "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" (UID: "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:21:12.894833 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.894752    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" (UID: "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:21:12.896516 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.896487    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" (UID: "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:21:12.896648 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.896541    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-kube-api-access-7hvlc" (OuterVolumeSpecName: "kube-api-access-7hvlc") pod "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" (UID: "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a"). InnerVolumeSpecName "kube-api-access-7hvlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:21:12.896648 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.896553    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" (UID: "a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:21:12.995521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.995495    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:21:12.995521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.995519    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-oauth-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:21:12.995694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.995532    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:21:12.995694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.995545    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hvlc\" (UniqueName: \"kubernetes.io/projected/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-kube-api-access-7hvlc\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:21:12.995694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.995558    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:21:12.995694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.995570    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-trusted-ca-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:21:12.995694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:12.995582    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a-console-oauth-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:21:13.564607 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:13.564564    2562 generic.go:356] "Generic (PLEG): container finished" podID="a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" containerID="56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437" exitCode=0
Sep 16 18:21:13.564983 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:13.564634    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-97c47c456-kjbzp" event={"ID":"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a","Type":"ContainerDied","Data":"56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437"}
Sep 16 18:21:13.564983 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:13.564709    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-97c47c456-kjbzp" event={"ID":"a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a","Type":"ContainerDied","Data":"3bab09f72ab01fb4a28db1105dc7cef080f0afcac432ab45d523f6ce086db991"}
Sep 16 18:21:13.564983 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:13.564647    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-97c47c456-kjbzp"
Sep 16 18:21:13.564983 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:13.564733    2562 scope.go:117] "RemoveContainer" containerID="56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437"
Sep 16 18:21:13.573324 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:13.573311    2562 scope.go:117] "RemoveContainer" containerID="56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437"
Sep 16 18:21:13.573601 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:21:13.573569    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437\": container with ID starting with 56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437 not found: ID does not exist" containerID="56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437"
Sep 16 18:21:13.573660 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:13.573630    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437"} err="failed to get container status \"56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437\": rpc error: code = NotFound desc = could not find container \"56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437\": container with ID starting with 56d99ae90cce16e4114394138172e92adf6a6e16771515d4af986d1eaccb1437 not found: ID does not exist"
Sep 16 18:21:13.585336 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:13.585311    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-97c47c456-kjbzp"]
Sep 16 18:21:13.589777 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:13.589757    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-97c47c456-kjbzp"]
Sep 16 18:21:14.285112 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:14.285085    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" path="/var/lib/kubelet/pods/a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a/volumes"
Sep 16 18:21:20.176884 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:20.176858    2562 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 18:21:53.941059 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:53.941018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:21:55.771052 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:21:55.771021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:21.792892 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:21.792858    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:21.887258 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:21.887225    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:25.814163 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:25.814131    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:25.920454 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:25.920423    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:28.023459 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:28.023419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:28.169684 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:28.169660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:28.300952 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:28.300892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:28.765264 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:28.765244    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:28.936164 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:28.936133    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:30.513520 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:30.513488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:30.909797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:30.909723    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:31.637654 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.637626    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rlk4l"]
Sep 16 18:22:31.638050 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.637914    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" containerName="console"
Sep 16 18:22:31.638050 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.637924    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" containerName="console"
Sep 16 18:22:31.638050 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.637971    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a52a4bd7-0ea0-4954-aac4-82c1eb6ec71a" containerName="console"
Sep 16 18:22:31.640748 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.640733    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l"
Sep 16 18:22:31.645098 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.645072    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 16 18:22:31.646123 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.646104    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-mxdkd\""
Sep 16 18:22:31.646246 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.646113    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 16 18:22:31.664486 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.664464    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rlk4l"]
Sep 16 18:22:31.814535 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.814508    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7bcc0eb2-8628-463b-94aa-9b7b96ed9576-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rlk4l\" (UID: \"7bcc0eb2-8628-463b-94aa-9b7b96ed9576\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l"
Sep 16 18:22:31.814672 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.814561    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhqsr\" (UniqueName: \"kubernetes.io/projected/7bcc0eb2-8628-463b-94aa-9b7b96ed9576-kube-api-access-qhqsr\") pod \"cert-manager-webhook-6486ffd997-rlk4l\" (UID: \"7bcc0eb2-8628-463b-94aa-9b7b96ed9576\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l"
Sep 16 18:22:31.915783 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.915762    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7bcc0eb2-8628-463b-94aa-9b7b96ed9576-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rlk4l\" (UID: \"7bcc0eb2-8628-463b-94aa-9b7b96ed9576\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l"
Sep 16 18:22:31.915864 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.915806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhqsr\" (UniqueName: \"kubernetes.io/projected/7bcc0eb2-8628-463b-94aa-9b7b96ed9576-kube-api-access-qhqsr\") pod \"cert-manager-webhook-6486ffd997-rlk4l\" (UID: \"7bcc0eb2-8628-463b-94aa-9b7b96ed9576\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l"
Sep 16 18:22:31.925225 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.925204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhqsr\" (UniqueName: \"kubernetes.io/projected/7bcc0eb2-8628-463b-94aa-9b7b96ed9576-kube-api-access-qhqsr\") pod \"cert-manager-webhook-6486ffd997-rlk4l\" (UID: \"7bcc0eb2-8628-463b-94aa-9b7b96ed9576\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l"
Sep 16 18:22:31.926027 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.926010    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7bcc0eb2-8628-463b-94aa-9b7b96ed9576-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rlk4l\" (UID: \"7bcc0eb2-8628-463b-94aa-9b7b96ed9576\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l"
Sep 16 18:22:31.949452 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:31.949431    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l"
Sep 16 18:22:32.227391 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:32.227350    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rlk4l"]
Sep 16 18:22:32.227570 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:22:32.227547    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bcc0eb2_8628_463b_94aa_9b7b96ed9576.slice/crio-3e5c5900a051b01f2b5a7815040059ff39bfe342245111266553dd9393f3123c WatchSource:0}: Error finding container 3e5c5900a051b01f2b5a7815040059ff39bfe342245111266553dd9393f3123c: Status 404 returned error can't find the container with id 3e5c5900a051b01f2b5a7815040059ff39bfe342245111266553dd9393f3123c
Sep 16 18:22:32.229569 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:32.229550    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:22:32.464916 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:32.464886    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:32.558922 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:32.558867    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:32.857736 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:32.857656    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l" event={"ID":"7bcc0eb2-8628-463b-94aa-9b7b96ed9576","Type":"ContainerStarted","Data":"3e5c5900a051b01f2b5a7815040059ff39bfe342245111266553dd9393f3123c"}
Sep 16 18:22:35.872528 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:35.872497    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l" event={"ID":"7bcc0eb2-8628-463b-94aa-9b7b96ed9576","Type":"ContainerStarted","Data":"ac427880ef2739ea0b637f1ee8029662ee00ca8f878125043ff65f816f0a9749"}
Sep 16 18:22:35.872885 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:35.872538    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l"
Sep 16 18:22:35.901710 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:35.901674    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l" podStartSLOduration=2.053269095 podStartE2EDuration="4.901660289s" totalImagesPullingTime="2.848391194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:22:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:22:35.899108819 +0000 UTC m=+376.138897571" watchObservedRunningTime="2026-09-16 18:22:35.901660289 +0000 UTC m=+376.141449040"
Sep 16 18:22:38.832926 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:38.832893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:39.336070 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:39.336042    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:39.448912 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:39.448880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:41.878195 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:41.878166    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rlk4l"
Sep 16 18:22:45.247473 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:45.247440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:45.351402 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:45.351368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:47.489369 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:47.489335    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:53.763093 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:53.763056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:22:53.828935 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:22:53.828903    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:03.295361 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:03.295326    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:03.414264 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:03.414233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:03.904672 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:03.904651    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:04.622440 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.622408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:04.721458 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.721433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:04.853821 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.853792    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"]
Sep 16 18:23:04.857074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.857059    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:04.861145 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.861128    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 18:23:04.866128 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.866110    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 18:23:04.886369 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.886306    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 18:23:04.886369 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.886333    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 18:23:04.886527 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.886488    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5ghg2\""
Sep 16 18:23:04.888437 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.888417    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"]
Sep 16 18:23:04.943653 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.943629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:04.943734 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.943673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87dlz\" (UniqueName: \"kubernetes.io/projected/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-kube-api-access-87dlz\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:04.943734 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.943714    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-tmp\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:04.943805 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.943732    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:04.943805 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:04.943747    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-manifests\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.030030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.030002    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:05.044194 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.044171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-tmp\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.044273 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.044200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.044273 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.044216    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-manifests\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.044359 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.044341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.044434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.044416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87dlz\" (UniqueName: \"kubernetes.io/projected/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-kube-api-access-87dlz\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.044656 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.044637    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-manifests\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.046565 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.046542    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-tmp\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.046897 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.046878    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-webhook-cert\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.046942 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.046883    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.053380 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.053352    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87dlz\" (UniqueName: \"kubernetes.io/projected/4187fb1c-8176-4c0f-82bb-ed212c9c1d41-kube-api-access-87dlz\") pod \"opendatahub-operator-controller-manager-86bdbbf975-2cjqf\" (UID: \"4187fb1c-8176-4c0f-82bb-ed212c9c1d41\") " pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.126071 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.126045    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:05.166456 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.166435    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:05.236699 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.236562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:05.344239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.344212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:05.461633 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.461519    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"]
Sep 16 18:23:05.464473 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:23:05.464452    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4187fb1c_8176_4c0f_82bb_ed212c9c1d41.slice/crio-53c4a56d05d1741bdad83b5eb87b2758f4559276f85ed50e6b9709abae47c581 WatchSource:0}: Error finding container 53c4a56d05d1741bdad83b5eb87b2758f4559276f85ed50e6b9709abae47c581: Status 404 returned error can't find the container with id 53c4a56d05d1741bdad83b5eb87b2758f4559276f85ed50e6b9709abae47c581
Sep 16 18:23:05.467023 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.466915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:05.569166 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.569140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:05.984196 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:05.984168    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf" event={"ID":"4187fb1c-8176-4c0f-82bb-ed212c9c1d41","Type":"ContainerStarted","Data":"53c4a56d05d1741bdad83b5eb87b2758f4559276f85ed50e6b9709abae47c581"}
Sep 16 18:23:09.000191 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:09.000166    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf" event={"ID":"4187fb1c-8176-4c0f-82bb-ed212c9c1d41","Type":"ContainerStarted","Data":"18171fb04d97cdddbd14c55670d4858c5b218ce55e6814281f515495c8f47373"}
Sep 16 18:23:10.006542 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:10.006477    2562 generic.go:356] "Generic (PLEG): container finished" podID="4187fb1c-8176-4c0f-82bb-ed212c9c1d41" containerID="18171fb04d97cdddbd14c55670d4858c5b218ce55e6814281f515495c8f47373" exitCode=0
Sep 16 18:23:10.006879 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:10.006546    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf" event={"ID":"4187fb1c-8176-4c0f-82bb-ed212c9c1d41","Type":"ContainerDied","Data":"18171fb04d97cdddbd14c55670d4858c5b218ce55e6814281f515495c8f47373"}
Sep 16 18:23:11.013171 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:11.013141    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf" event={"ID":"4187fb1c-8176-4c0f-82bb-ed212c9c1d41","Type":"ContainerStarted","Data":"5f25eb12a6c6ee96cc907fdf393e03489e293d3cc3f97b3b5b7fc95776a9ece7"}
Sep 16 18:23:11.013518 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:11.013268    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:11.040702 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:11.040661    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf" podStartSLOduration=2.6189009309999998 podStartE2EDuration="7.040646379s" totalImagesPullingTime="3.471797136s" totalInitContainerRuntime="949.948312ms" isStatefulPod=false podCreationTimestamp="2026-09-16 18:23:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:23:11.038366724 +0000 UTC m=+411.278155475" watchObservedRunningTime="2026-09-16 18:23:11.040646379 +0000 UTC m=+411.280435131"
Sep 16 18:23:13.840211 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:13.840172    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:13.931378 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:13.931348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:19.761558 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:19.761526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:19.822999 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:19.822968    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:20.191811 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:20.191783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:20.251403 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:20.251377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:22.018285 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:22.018257    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86bdbbf975-2cjqf"
Sep 16 18:23:26.006047 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.006018    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66fb54f4b8-xl7gs"]
Sep 16 18:23:26.014709 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.014689    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.022197 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.022176    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66fb54f4b8-xl7gs"]
Sep 16 18:23:26.074413 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.074386    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:26.100580 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.100560    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c5jr\" (UniqueName: \"kubernetes.io/projected/4d7c3815-73b2-408b-9525-560443edd089-kube-api-access-5c5jr\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.100682 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.100608    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d7c3815-73b2-408b-9525-560443edd089-console-serving-cert\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.100682 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.100636    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d7c3815-73b2-408b-9525-560443edd089-console-oauth-config\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.100682 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.100661    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-service-ca\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.100788 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.100715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-trusted-ca-bundle\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.100788 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.100738    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-oauth-serving-cert\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.100788 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.100773    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-console-config\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.201266 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.201241    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-trusted-ca-bundle\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.201354 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.201270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-oauth-serving-cert\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.201354 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.201308    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-console-config\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.201433 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.201349    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c5jr\" (UniqueName: \"kubernetes.io/projected/4d7c3815-73b2-408b-9525-560443edd089-kube-api-access-5c5jr\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.201433 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.201380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d7c3815-73b2-408b-9525-560443edd089-console-serving-cert\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.201433 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.201408    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d7c3815-73b2-408b-9525-560443edd089-console-oauth-config\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.201569 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.201431    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-service-ca\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.202039 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.202017    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-oauth-serving-cert\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.202135 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.202049    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-console-config\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.202135 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.202119    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-service-ca\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.202222 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.202140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d7c3815-73b2-408b-9525-560443edd089-trusted-ca-bundle\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.203941 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.203918    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d7c3815-73b2-408b-9525-560443edd089-console-serving-cert\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.204022 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.203945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d7c3815-73b2-408b-9525-560443edd089-console-oauth-config\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.209862 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.209840    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c5jr\" (UniqueName: \"kubernetes.io/projected/4d7c3815-73b2-408b-9525-560443edd089-kube-api-access-5c5jr\") pod \"console-66fb54f4b8-xl7gs\" (UID: \"4d7c3815-73b2-408b-9525-560443edd089\") " pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.324142 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.324096    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:26.598416 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:26.598369    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66fb54f4b8-xl7gs"]
Sep 16 18:23:26.600584 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:23:26.600557    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d7c3815_73b2_408b_9525_560443edd089.slice/crio-f94bd34b71b199bd7d463c7bf4af0b6a0178efc6682f92873da16ebfbfe97fbf WatchSource:0}: Error finding container f94bd34b71b199bd7d463c7bf4af0b6a0178efc6682f92873da16ebfbfe97fbf: Status 404 returned error can't find the container with id f94bd34b71b199bd7d463c7bf4af0b6a0178efc6682f92873da16ebfbfe97fbf
Sep 16 18:23:27.077704 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:27.077672    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66fb54f4b8-xl7gs" event={"ID":"4d7c3815-73b2-408b-9525-560443edd089","Type":"ContainerStarted","Data":"d9649009e4253be252b63af18f55a4ece586bbb58508286c900e09da27d52f09"}
Sep 16 18:23:27.078125 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:27.077767    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66fb54f4b8-xl7gs" event={"ID":"4d7c3815-73b2-408b-9525-560443edd089","Type":"ContainerStarted","Data":"f94bd34b71b199bd7d463c7bf4af0b6a0178efc6682f92873da16ebfbfe97fbf"}
Sep 16 18:23:27.110637 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:27.110574    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66fb54f4b8-xl7gs" podStartSLOduration=2.11056101 podStartE2EDuration="2.11056101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:23:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:23:27.108452606 +0000 UTC m=+427.348241357" watchObservedRunningTime="2026-09-16 18:23:27.11056101 +0000 UTC m=+427.350349761"
Sep 16 18:23:27.258297 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:27.258268    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:27.363673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:27.363557    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:32.871934 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:32.871902    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:32.943316 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:32.943287    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"]
Sep 16 18:23:32.947197 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:32.947182    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:32.949872 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:32.949851    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 18:23:32.949872 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:32.949869    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wfr2t\""
Sep 16 18:23:32.950032 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:32.949993    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 18:23:32.962456 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:32.962434    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"]
Sep 16 18:23:33.049286 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.049260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2mqn\" (UniqueName: \"kubernetes.io/projected/a4436fe7-191c-4c7d-b381-e7525adb6680-kube-api-access-p2mqn\") pod \"kube-auth-proxy-7d9c67857d-p97jv\" (UID: \"a4436fe7-191c-4c7d-b381-e7525adb6680\") " pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:33.049382 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.049301    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4436fe7-191c-4c7d-b381-e7525adb6680-tls-certs\") pod \"kube-auth-proxy-7d9c67857d-p97jv\" (UID: \"a4436fe7-191c-4c7d-b381-e7525adb6680\") " pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:33.049430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.049382    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4436fe7-191c-4c7d-b381-e7525adb6680-tmp\") pod \"kube-auth-proxy-7d9c67857d-p97jv\" (UID: \"a4436fe7-191c-4c7d-b381-e7525adb6680\") " pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:33.149804 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.149753    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4436fe7-191c-4c7d-b381-e7525adb6680-tls-certs\") pod \"kube-auth-proxy-7d9c67857d-p97jv\" (UID: \"a4436fe7-191c-4c7d-b381-e7525adb6680\") " pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:33.149887 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.149806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4436fe7-191c-4c7d-b381-e7525adb6680-tmp\") pod \"kube-auth-proxy-7d9c67857d-p97jv\" (UID: \"a4436fe7-191c-4c7d-b381-e7525adb6680\") " pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:33.149887 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.149838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2mqn\" (UniqueName: \"kubernetes.io/projected/a4436fe7-191c-4c7d-b381-e7525adb6680-kube-api-access-p2mqn\") pod \"kube-auth-proxy-7d9c67857d-p97jv\" (UID: \"a4436fe7-191c-4c7d-b381-e7525adb6680\") " pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:33.149887 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:23:33.149879    2562 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 16 18:23:33.150014 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:23:33.149940    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4436fe7-191c-4c7d-b381-e7525adb6680-tls-certs podName:a4436fe7-191c-4c7d-b381-e7525adb6680 nodeName:}" failed. No retries permitted until 2026-09-16 18:23:33.64992657 +0000 UTC m=+433.889715300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a4436fe7-191c-4c7d-b381-e7525adb6680-tls-certs") pod "kube-auth-proxy-7d9c67857d-p97jv" (UID: "a4436fe7-191c-4c7d-b381-e7525adb6680") : secret "kube-auth-proxy-tls" not found
Sep 16 18:23:33.152078 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.152060    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4436fe7-191c-4c7d-b381-e7525adb6680-tmp\") pod \"kube-auth-proxy-7d9c67857d-p97jv\" (UID: \"a4436fe7-191c-4c7d-b381-e7525adb6680\") " pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:33.160931 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.160913    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2mqn\" (UniqueName: \"kubernetes.io/projected/a4436fe7-191c-4c7d-b381-e7525adb6680-kube-api-access-p2mqn\") pod \"kube-auth-proxy-7d9c67857d-p97jv\" (UID: \"a4436fe7-191c-4c7d-b381-e7525adb6680\") " pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:33.567569 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.567541    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:33.653883 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.653854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4436fe7-191c-4c7d-b381-e7525adb6680-tls-certs\") pod \"kube-auth-proxy-7d9c67857d-p97jv\" (UID: \"a4436fe7-191c-4c7d-b381-e7525adb6680\") " pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:33.656141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.656122    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4436fe7-191c-4c7d-b381-e7525adb6680-tls-certs\") pod \"kube-auth-proxy-7d9c67857d-p97jv\" (UID: \"a4436fe7-191c-4c7d-b381-e7525adb6680\") " pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:33.856626 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:33.856564    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"
Sep 16 18:23:34.144967 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.144901    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv"]
Sep 16 18:23:34.147877 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:23:34.147852    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4436fe7_191c_4c7d_b381_e7525adb6680.slice/crio-beb4c2dcc1351ab199caec914429b75edcb1a4be6cacc0931230dc70794c9bcf WatchSource:0}: Error finding container beb4c2dcc1351ab199caec914429b75edcb1a4be6cacc0931230dc70794c9bcf: Status 404 returned error can't find the container with id beb4c2dcc1351ab199caec914429b75edcb1a4be6cacc0931230dc70794c9bcf
Sep 16 18:23:34.160700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.160674    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:34.177771 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.177748    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"]
Sep 16 18:23:34.182037 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.182019    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.184644 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.184620    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 18:23:34.184726 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.184659    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 16 18:23:34.184726 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.184663    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7g9l9\""
Sep 16 18:23:34.184910 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.184896    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 16 18:23:34.185110 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.185095    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 16 18:23:34.200910 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.200885    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"]
Sep 16 18:23:34.233445 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.233418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:34.258700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.258678    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hf8d\" (UniqueName: \"kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-kube-api-access-5hf8d\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.258804 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.258704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.258804 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.258722    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.258873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.258805    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.258873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.258843    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.258936 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.258871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/26dc2484-432a-44c4-897c-ced8abcfdf2d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.258936 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.258887    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.360178 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.360155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hf8d\" (UniqueName: \"kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-kube-api-access-5hf8d\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.360275 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.360187    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.360275 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.360203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.360275 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.360257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.360429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.360294    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.360429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.360326    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/26dc2484-432a-44c4-897c-ced8abcfdf2d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.360429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.360349    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.361234 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.361211    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.362685 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.362661    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/26dc2484-432a-44c4-897c-ced8abcfdf2d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.362779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.362721    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.362866 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.362847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.362917 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.362861    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.368722 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.368704    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.368960 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.368945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hf8d\" (UniqueName: \"kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-kube-api-access-5hf8d\") pod \"istiod-openshift-gateway-7b7995d5cd-frpld\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.490844 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.490815    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:34.766161 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:34.766133    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"]
Sep 16 18:23:34.768427 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:23:34.768399    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26dc2484_432a_44c4_897c_ced8abcfdf2d.slice/crio-abd07018963b323cdc7c4eb408082373c9d128af89d42396e02a9708c712a7c0 WatchSource:0}: Error finding container abd07018963b323cdc7c4eb408082373c9d128af89d42396e02a9708c712a7c0: Status 404 returned error can't find the container with id abd07018963b323cdc7c4eb408082373c9d128af89d42396e02a9708c712a7c0
Sep 16 18:23:35.114688 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:35.114611    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld" event={"ID":"26dc2484-432a-44c4-897c-ced8abcfdf2d","Type":"ContainerStarted","Data":"abd07018963b323cdc7c4eb408082373c9d128af89d42396e02a9708c712a7c0"}
Sep 16 18:23:35.116253 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:35.116223    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv" event={"ID":"a4436fe7-191c-4c7d-b381-e7525adb6680","Type":"ContainerStarted","Data":"beb4c2dcc1351ab199caec914429b75edcb1a4be6cacc0931230dc70794c9bcf"}
Sep 16 18:23:35.288791 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:35.288764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:36.325006 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:36.324970    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:36.325442 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:36.325028    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:36.330568 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:36.330547    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:37.129044 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:37.128962    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv" event={"ID":"a4436fe7-191c-4c7d-b381-e7525adb6680","Type":"ContainerStarted","Data":"f44b5cbb99c585f45583f97fb6afb9a60b2642d3341453488e99939b7aa6936a"}
Sep 16 18:23:37.134213 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:37.134185    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66fb54f4b8-xl7gs"
Sep 16 18:23:37.148377 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:37.148322    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d9c67857d-p97jv" podStartSLOduration=2.426691248 podStartE2EDuration="5.148303384s" totalImagesPullingTime="2.721612136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:23:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:23:37.146883837 +0000 UTC m=+437.386672589" watchObservedRunningTime="2026-09-16 18:23:37.148303384 +0000 UTC m=+437.388092137"
Sep 16 18:23:37.199851 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:37.199811    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d5db6bb8-g8k2p"]
Sep 16 18:23:37.356868 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:37.356838    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:37.457666 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:37.457628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:37.518411 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:37.518375    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:38.256392 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:38.256357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:38.357315 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:38.357281    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:38.537569 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:38.537485    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:38.720397 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:38.720357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:38.822621 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:38.822495    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:38.903557 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:38.903523    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:39.367558 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:39.367513    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:39.544571 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:39.544537    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 16 18:23:39.544671 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:39.544628    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 16 18:23:40.144628 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:40.144581    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld" event={"ID":"26dc2484-432a-44c4-897c-ced8abcfdf2d","Type":"ContainerStarted","Data":"9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47"}
Sep 16 18:23:40.144834 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:40.144816    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:40.146184 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:40.146160    2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-frpld container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 18:23:40.146290 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:40.146211    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld" podUID="26dc2484-432a-44c4-897c-ced8abcfdf2d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:23:40.167540 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:40.167489    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld" podStartSLOduration=1.39390779 podStartE2EDuration="6.167471271s" totalImagesPullingTime="4.773563481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:23:40.16607789 +0000 UTC m=+440.405866679" watchObservedRunningTime="2026-09-16 18:23:40.167471271 +0000 UTC m=+440.407260024"
Sep 16 18:23:40.434084 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:40.434050    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:41.150386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:41.150364    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:41.221749 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:41.221718    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:41.456116 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:41.456089    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:41.794202 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:41.794107    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:47.409717 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.409683    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"]
Sep 16 18:23:47.413625 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.413603    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.421781 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.421761    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 18:23:47.422246 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.422230    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lg7l8\""
Sep 16 18:23:47.423460 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.423444    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 18:23:47.423753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.423738    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 18:23:47.425002 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.424986    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:23:47.425518 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.425500    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 18:23:47.503363 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.503341    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"]
Sep 16 18:23:47.568640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.568621    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15b621ee-8544-44b2-b121-c326440878d4-cert\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.568753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.568658    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15b621ee-8544-44b2-b121-c326440878d4-manager-config\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.568753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.568685    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wltfp\" (UniqueName: \"kubernetes.io/projected/15b621ee-8544-44b2-b121-c326440878d4-kube-api-access-wltfp\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.568863 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.568766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15b621ee-8544-44b2-b121-c326440878d4-metrics-cert\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.669301 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.669265    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15b621ee-8544-44b2-b121-c326440878d4-cert\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.669474 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.669334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15b621ee-8544-44b2-b121-c326440878d4-manager-config\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.669474 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.669376    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wltfp\" (UniqueName: \"kubernetes.io/projected/15b621ee-8544-44b2-b121-c326440878d4-kube-api-access-wltfp\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.669474 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.669400    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15b621ee-8544-44b2-b121-c326440878d4-metrics-cert\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.670138 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.670112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/15b621ee-8544-44b2-b121-c326440878d4-manager-config\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.672138 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.672094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/15b621ee-8544-44b2-b121-c326440878d4-cert\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.672239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.672201    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/15b621ee-8544-44b2-b121-c326440878d4-metrics-cert\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.683116 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.683095    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wltfp\" (UniqueName: \"kubernetes.io/projected/15b621ee-8544-44b2-b121-c326440878d4-kube-api-access-wltfp\") pod \"lws-controller-manager-6fb494f6fb-khrnj\" (UID: \"15b621ee-8544-44b2-b121-c326440878d4\") " pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:47.724012 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:47.723990    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:48.011135 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:48.011076    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"]
Sep 16 18:23:48.013224 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:23:48.013201    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b621ee_8544_44b2_b121_c326440878d4.slice/crio-d30695a5036ff646bab1d98230b58130e183ae25309c989930e3e6b00fecd22d WatchSource:0}: Error finding container d30695a5036ff646bab1d98230b58130e183ae25309c989930e3e6b00fecd22d: Status 404 returned error can't find the container with id d30695a5036ff646bab1d98230b58130e183ae25309c989930e3e6b00fecd22d
Sep 16 18:23:48.178830 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:48.178806    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj" event={"ID":"15b621ee-8544-44b2-b121-c326440878d4","Type":"ContainerStarted","Data":"d30695a5036ff646bab1d98230b58130e183ae25309c989930e3e6b00fecd22d"}
Sep 16 18:23:48.966634 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:48.966607    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:49.063761 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:49.063729    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:49.161610 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:49.161488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:49.326746 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:49.326571    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:49.424538 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:49.424447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:49.544208 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:49.544134    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:49.648171 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:49.648088    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:49.823435 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:49.823405    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:23:51.200157 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:51.200125    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj" event={"ID":"15b621ee-8544-44b2-b121-c326440878d4","Type":"ContainerStarted","Data":"e160ca708b8ca25ca752e818315f120811e1fd58947634293b554301d55daee2"}
Sep 16 18:23:51.200529 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:51.200243    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:23:51.225327 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:51.225290    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj" podStartSLOduration=1.384873349 podStartE2EDuration="4.225278041s" totalImagesPullingTime="2.840404692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:23:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:23:51.224143235 +0000 UTC m=+451.463931987" watchObservedRunningTime="2026-09-16 18:23:51.225278041 +0000 UTC m=+451.465066827"
Sep 16 18:23:52.795125 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:52.795097    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"]
Sep 16 18:23:52.795568 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:52.795265    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld" podUID="26dc2484-432a-44c4-897c-ced8abcfdf2d" containerName="discovery" containerID="cri-o://9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47" gracePeriod=30
Sep 16 18:23:53.037041 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.037019    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:53.112562 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.112499    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hf8d\" (UniqueName: \"kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-kube-api-access-5hf8d\") pod \"26dc2484-432a-44c4-897c-ced8abcfdf2d\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") "
Sep 16 18:23:53.112562 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.112543    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-kubeconfig\") pod \"26dc2484-432a-44c4-897c-ced8abcfdf2d\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") "
Sep 16 18:23:53.112761 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.112578    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/26dc2484-432a-44c4-897c-ced8abcfdf2d-local-certs\") pod \"26dc2484-432a-44c4-897c-ced8abcfdf2d\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") "
Sep 16 18:23:53.112761 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.112661    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-cacerts\") pod \"26dc2484-432a-44c4-897c-ced8abcfdf2d\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") "
Sep 16 18:23:53.112761 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.112726    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-token\") pod \"26dc2484-432a-44c4-897c-ced8abcfdf2d\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") "
Sep 16 18:23:53.112761 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.112753    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-ca-configmap\") pod \"26dc2484-432a-44c4-897c-ced8abcfdf2d\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") "
Sep 16 18:23:53.112943 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.112776    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-dns-cert\") pod \"26dc2484-432a-44c4-897c-ced8abcfdf2d\" (UID: \"26dc2484-432a-44c4-897c-ced8abcfdf2d\") "
Sep 16 18:23:53.113107 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.113079    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "26dc2484-432a-44c4-897c-ced8abcfdf2d" (UID: "26dc2484-432a-44c4-897c-ced8abcfdf2d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:23:53.114925 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.114901    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-kube-api-access-5hf8d" (OuterVolumeSpecName: "kube-api-access-5hf8d") pod "26dc2484-432a-44c4-897c-ced8abcfdf2d" (UID: "26dc2484-432a-44c4-897c-ced8abcfdf2d"). InnerVolumeSpecName "kube-api-access-5hf8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:23:53.115211 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.115185    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "26dc2484-432a-44c4-897c-ced8abcfdf2d" (UID: "26dc2484-432a-44c4-897c-ced8abcfdf2d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:23:53.115313 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.115209    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "26dc2484-432a-44c4-897c-ced8abcfdf2d" (UID: "26dc2484-432a-44c4-897c-ced8abcfdf2d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:23:53.115313 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.115245    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26dc2484-432a-44c4-897c-ced8abcfdf2d-local-certs" (OuterVolumeSpecName: "local-certs") pod "26dc2484-432a-44c4-897c-ced8abcfdf2d" (UID: "26dc2484-432a-44c4-897c-ced8abcfdf2d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:23:53.115313 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.115285    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-token" (OuterVolumeSpecName: "istio-token") pod "26dc2484-432a-44c4-897c-ced8abcfdf2d" (UID: "26dc2484-432a-44c4-897c-ced8abcfdf2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:23:53.115313 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.115302    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-cacerts" (OuterVolumeSpecName: "cacerts") pod "26dc2484-432a-44c4-897c-ced8abcfdf2d" (UID: "26dc2484-432a-44c4-897c-ced8abcfdf2d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:23:53.210755 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.210732    2562 generic.go:356] "Generic (PLEG): container finished" podID="26dc2484-432a-44c4-897c-ced8abcfdf2d" containerID="9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47" exitCode=0
Sep 16 18:23:53.210847 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.210772    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"
Sep 16 18:23:53.210847 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.210819    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld" event={"ID":"26dc2484-432a-44c4-897c-ced8abcfdf2d","Type":"ContainerDied","Data":"9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47"}
Sep 16 18:23:53.210912 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.210860    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld" event={"ID":"26dc2484-432a-44c4-897c-ced8abcfdf2d","Type":"ContainerDied","Data":"abd07018963b323cdc7c4eb408082373c9d128af89d42396e02a9708c712a7c0"}
Sep 16 18:23:53.210912 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.210879    2562 scope.go:117] "RemoveContainer" containerID="9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47"
Sep 16 18:23:53.213342 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.213321    2562 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-cacerts\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:23:53.213424 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.213344    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-token\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:23:53.213424 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.213354    2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-ca-configmap\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:23:53.213424 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.213362    2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-csr-dns-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:23:53.213424 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.213370    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hf8d\" (UniqueName: \"kubernetes.io/projected/26dc2484-432a-44c4-897c-ced8abcfdf2d-kube-api-access-5hf8d\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:23:53.213424 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.213378    2562 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26dc2484-432a-44c4-897c-ced8abcfdf2d-istio-kubeconfig\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:23:53.213424 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.213386    2562 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/26dc2484-432a-44c4-897c-ced8abcfdf2d-local-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:23:53.219394 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.219370    2562 scope.go:117] "RemoveContainer" containerID="9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47"
Sep 16 18:23:53.219649 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:23:53.219632    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47\": container with ID starting with 9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47 not found: ID does not exist" containerID="9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47"
Sep 16 18:23:53.219710 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.219660    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47"} err="failed to get container status \"9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47\": rpc error: code = NotFound desc = could not find container \"9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47\": container with ID starting with 9441e05e725520fc10917152f6b35012fb8ae78a2476f25baf15dd516e0c4d47 not found: ID does not exist"
Sep 16 18:23:53.250787 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.250766    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"]
Sep 16 18:23:53.255872 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:53.255852    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-frpld"]
Sep 16 18:23:54.284921 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:23:54.284886    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26dc2484-432a-44c4-897c-ced8abcfdf2d" path="/var/lib/kubelet/pods/26dc2484-432a-44c4-897c-ced8abcfdf2d/volumes"
Sep 16 18:24:02.206901 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.206870    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6fb494f6fb-khrnj"
Sep 16 18:24:02.227713 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.227685    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d5db6bb8-g8k2p" podUID="82c2b462-4cc1-4a13-85a5-2b536f0e04ba" containerName="console" containerID="cri-o://90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5" gracePeriod=15
Sep 16 18:24:02.472319 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.472299    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:24:02.579239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.579212    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-serving-cert\") pod \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") "
Sep 16 18:24:02.579352 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.579253    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmqbg\" (UniqueName: \"kubernetes.io/projected/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-kube-api-access-vmqbg\") pod \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") "
Sep 16 18:24:02.579352 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.579285    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-oauth-config\") pod \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") "
Sep 16 18:24:02.579352 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.579345    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-oauth-serving-cert\") pod \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") "
Sep 16 18:24:02.579514 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.579367    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-trusted-ca-bundle\") pod \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") "
Sep 16 18:24:02.579514 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.579404    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-config\") pod \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") "
Sep 16 18:24:02.579514 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.579461    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-service-ca\") pod \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\" (UID: \"82c2b462-4cc1-4a13-85a5-2b536f0e04ba\") "
Sep 16 18:24:02.579838 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.579796    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "82c2b462-4cc1-4a13-85a5-2b536f0e04ba" (UID: "82c2b462-4cc1-4a13-85a5-2b536f0e04ba"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:24:02.579957 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.579886    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-service-ca" (OuterVolumeSpecName: "service-ca") pod "82c2b462-4cc1-4a13-85a5-2b536f0e04ba" (UID: "82c2b462-4cc1-4a13-85a5-2b536f0e04ba"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:24:02.580184 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.579972    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-config" (OuterVolumeSpecName: "console-config") pod "82c2b462-4cc1-4a13-85a5-2b536f0e04ba" (UID: "82c2b462-4cc1-4a13-85a5-2b536f0e04ba"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:24:02.580184 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.580029    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "82c2b462-4cc1-4a13-85a5-2b536f0e04ba" (UID: "82c2b462-4cc1-4a13-85a5-2b536f0e04ba"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:24:02.581389 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.581367    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "82c2b462-4cc1-4a13-85a5-2b536f0e04ba" (UID: "82c2b462-4cc1-4a13-85a5-2b536f0e04ba"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:24:02.581468 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.581439    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "82c2b462-4cc1-4a13-85a5-2b536f0e04ba" (UID: "82c2b462-4cc1-4a13-85a5-2b536f0e04ba"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:24:02.581510 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.581471    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-kube-api-access-vmqbg" (OuterVolumeSpecName: "kube-api-access-vmqbg") pod "82c2b462-4cc1-4a13-85a5-2b536f0e04ba" (UID: "82c2b462-4cc1-4a13-85a5-2b536f0e04ba"). InnerVolumeSpecName "kube-api-access-vmqbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:24:02.680119 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.680097    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:24:02.680119 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.680115    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:24:02.680231 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.680124    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmqbg\" (UniqueName: \"kubernetes.io/projected/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-kube-api-access-vmqbg\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:24:02.680231 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.680132    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-oauth-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:24:02.680231 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.680140    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-oauth-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:24:02.680231 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.680148    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-trusted-ca-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:24:02.680231 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:02.680155    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82c2b462-4cc1-4a13-85a5-2b536f0e04ba-console-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:24:03.253840 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:03.253813    2562 generic.go:356] "Generic (PLEG): container finished" podID="82c2b462-4cc1-4a13-85a5-2b536f0e04ba" containerID="90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5" exitCode=0
Sep 16 18:24:03.254129 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:03.253861    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5db6bb8-g8k2p"
Sep 16 18:24:03.254129 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:03.253894    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5db6bb8-g8k2p" event={"ID":"82c2b462-4cc1-4a13-85a5-2b536f0e04ba","Type":"ContainerDied","Data":"90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5"}
Sep 16 18:24:03.254129 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:03.253939    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5db6bb8-g8k2p" event={"ID":"82c2b462-4cc1-4a13-85a5-2b536f0e04ba","Type":"ContainerDied","Data":"236e4b6569645c46fa0e2a709f20d51e848722122da9276440dbf9a9afe56d46"}
Sep 16 18:24:03.254129 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:03.253958    2562 scope.go:117] "RemoveContainer" containerID="90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5"
Sep 16 18:24:03.262471 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:03.262456    2562 scope.go:117] "RemoveContainer" containerID="90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5"
Sep 16 18:24:03.262737 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:24:03.262719    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5\": container with ID starting with 90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5 not found: ID does not exist" containerID="90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5"
Sep 16 18:24:03.262791 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:03.262743    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5"} err="failed to get container status \"90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5\": rpc error: code = NotFound desc = could not find container \"90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5\": container with ID starting with 90ef778452610a5cdc69bd65b409b9b298ea5351b87d701ff9fe4cd74d5ed1b5 not found: ID does not exist"
Sep 16 18:24:03.273668 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:03.273645    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d5db6bb8-g8k2p"]
Sep 16 18:24:03.277719 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:03.277702    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d5db6bb8-g8k2p"]
Sep 16 18:24:04.283920 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:04.283896    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82c2b462-4cc1-4a13-85a5-2b536f0e04ba" path="/var/lib/kubelet/pods/82c2b462-4cc1-4a13-85a5-2b536f0e04ba/volumes"
Sep 16 18:24:26.187841 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:26.187809    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:24:39.325147 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:39.325112    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:24:39.432960 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:24:39.432932    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:00.090693 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:00.090663    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:00.888292 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:00.888262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:03.774966 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:03.774936    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:03.940117 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:03.940077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:04.514561 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:04.514529    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:04.927227 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:04.927194    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:06.178511 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:06.178479    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:06.292758 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:06.292729    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:08.292510 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:08.292464    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:08.937969 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:08.937941    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:10.523349 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:10.523317    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:11.538422 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:11.538395    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:13.507261 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:13.507233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:13.609931 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:13.609905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:13.902604 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:13.902508    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:13.960164 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:13.960137    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:19.699176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:19.699142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:19.761054 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:19.761025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:19.904082 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:19.904055    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:19.968796 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:19.968737    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:20.099627 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:20.099584    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:20.159630 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:20.159607    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:21.212145 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.212114    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:21.216081 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.216046    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"]
Sep 16 18:25:21.216463 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.216445    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82c2b462-4cc1-4a13-85a5-2b536f0e04ba" containerName="console"
Sep 16 18:25:21.216618 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.216465    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="82c2b462-4cc1-4a13-85a5-2b536f0e04ba" containerName="console"
Sep 16 18:25:21.216618 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.216496    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26dc2484-432a-44c4-897c-ced8abcfdf2d" containerName="discovery"
Sep 16 18:25:21.216618 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.216505    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="26dc2484-432a-44c4-897c-ced8abcfdf2d" containerName="discovery"
Sep 16 18:25:21.216618 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.216607    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="82c2b462-4cc1-4a13-85a5-2b536f0e04ba" containerName="console"
Sep 16 18:25:21.216618 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.216619    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="26dc2484-432a-44c4-897c-ced8abcfdf2d" containerName="discovery"
Sep 16 18:25:21.219530 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.219513    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"
Sep 16 18:25:21.221850 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.221832    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 16 18:25:21.221946 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.221905    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 18:25:21.222111 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.222096    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 16 18:25:21.222200 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.222104    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 18:25:21.222940 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.222923    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-57fd7\""
Sep 16 18:25:21.228390 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.228371    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"]
Sep 16 18:25:21.312908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.312887    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/84176ad5-47ab-45c8-af4b-e001945f01a7-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-msrll\" (UID: \"84176ad5-47ab-45c8-af4b-e001945f01a7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"
Sep 16 18:25:21.312998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.312926    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/84176ad5-47ab-45c8-af4b-e001945f01a7-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-msrll\" (UID: \"84176ad5-47ab-45c8-af4b-e001945f01a7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"
Sep 16 18:25:21.312998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.312949    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bf2v\" (UniqueName: \"kubernetes.io/projected/84176ad5-47ab-45c8-af4b-e001945f01a7-kube-api-access-9bf2v\") pod \"kuadrant-console-plugin-fbdd4f445-msrll\" (UID: \"84176ad5-47ab-45c8-af4b-e001945f01a7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"
Sep 16 18:25:21.398540 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.398510    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:21.413697 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.413676    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bf2v\" (UniqueName: \"kubernetes.io/projected/84176ad5-47ab-45c8-af4b-e001945f01a7-kube-api-access-9bf2v\") pod \"kuadrant-console-plugin-fbdd4f445-msrll\" (UID: \"84176ad5-47ab-45c8-af4b-e001945f01a7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"
Sep 16 18:25:21.413787 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.413730    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/84176ad5-47ab-45c8-af4b-e001945f01a7-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-msrll\" (UID: \"84176ad5-47ab-45c8-af4b-e001945f01a7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"
Sep 16 18:25:21.413787 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.413767    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/84176ad5-47ab-45c8-af4b-e001945f01a7-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-msrll\" (UID: \"84176ad5-47ab-45c8-af4b-e001945f01a7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"
Sep 16 18:25:21.414266 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.414241    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/84176ad5-47ab-45c8-af4b-e001945f01a7-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-msrll\" (UID: \"84176ad5-47ab-45c8-af4b-e001945f01a7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"
Sep 16 18:25:21.416225 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.416207    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/84176ad5-47ab-45c8-af4b-e001945f01a7-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-msrll\" (UID: \"84176ad5-47ab-45c8-af4b-e001945f01a7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"
Sep 16 18:25:21.422873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.422855    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bf2v\" (UniqueName: \"kubernetes.io/projected/84176ad5-47ab-45c8-af4b-e001945f01a7-kube-api-access-9bf2v\") pod \"kuadrant-console-plugin-fbdd4f445-msrll\" (UID: \"84176ad5-47ab-45c8-af4b-e001945f01a7\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"
Sep 16 18:25:21.529161 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.529110    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"
Sep 16 18:25:21.800655 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:21.800635    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll"]
Sep 16 18:25:21.802815 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:25:21.802783    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84176ad5_47ab_45c8_af4b_e001945f01a7.slice/crio-1e95316b5f412a107f76a73fc1bfe750fe9d5f3c24f4bbf07b71d82fda62b960 WatchSource:0}: Error finding container 1e95316b5f412a107f76a73fc1bfe750fe9d5f3c24f4bbf07b71d82fda62b960: Status 404 returned error can't find the container with id 1e95316b5f412a107f76a73fc1bfe750fe9d5f3c24f4bbf07b71d82fda62b960
Sep 16 18:25:22.567813 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:22.567781    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll" event={"ID":"84176ad5-47ab-45c8-af4b-e001945f01a7","Type":"ContainerStarted","Data":"1e95316b5f412a107f76a73fc1bfe750fe9d5f3c24f4bbf07b71d82fda62b960"}
Sep 16 18:25:26.117320 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:26.117240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:26.218433 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:26.218302    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:26.593211 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:26.593185    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll" event={"ID":"84176ad5-47ab-45c8-af4b-e001945f01a7","Type":"ContainerStarted","Data":"60fa8a1c5a976ce3f3b7fed4280501b22e9f6971a029b8eb21504b1231bf1fa6"}
Sep 16 18:25:26.611152 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:26.611102    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-msrll" podStartSLOduration=1.104945801 podStartE2EDuration="5.611085003s" totalImagesPullingTime="4.506139202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:25:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:25:26.610284194 +0000 UTC m=+546.850072957" watchObservedRunningTime="2026-09-16 18:25:26.611085003 +0000 UTC m=+546.850873755"
Sep 16 18:25:28.574312 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:28.574248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:28.675700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:28.675673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:33.339172 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:33.339121    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:33.431661 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:33.431629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:33.533041 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:33.533014    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:38.721100 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:38.721060    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:38.783685 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:38.783660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:39.114671 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:39.114614    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:39.174683 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:39.174654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:39.518447 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:39.518404    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:39.581727 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:39.581698    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:45.538174 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:45.538143    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:45.642511 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:45.642454    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:48.003602 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:48.003566    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:48.108052 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:48.108012    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:25:57.629783 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:25:57.629748    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:03.667894 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:03.667858    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:03.986755 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:03.986670    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:14.414145 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:14.414110    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:14.514479 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:14.514455    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:19.752600 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:19.752544    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:19.860143 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:19.860100    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:19.913467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:19.913435    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:20.547759 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.547729    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-s47pl"]
Sep 16 18:26:20.550196 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.550178    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl"
Sep 16 18:26:20.552294 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.552274    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vh4db\""
Sep 16 18:26:20.557472 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.557453    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-s47pl"]
Sep 16 18:26:20.586220 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.586196    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:20.645286 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.645265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpkhz\" (UniqueName: \"kubernetes.io/projected/012c1b4f-5e26-49f2-a9b6-2792ec298503-kube-api-access-hpkhz\") pod \"authorino-7bcfd8f9cd-s47pl\" (UID: \"012c1b4f-5e26-49f2-a9b6-2792ec298503\") " pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl"
Sep 16 18:26:20.658203 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.658182    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-gbw6z"]
Sep 16 18:26:20.660161 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.660146    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-gbw6z"
Sep 16 18:26:20.661988 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.661962    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:20.670762 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.670740    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-gbw6z"]
Sep 16 18:26:20.745598 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.745574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpkhz\" (UniqueName: \"kubernetes.io/projected/012c1b4f-5e26-49f2-a9b6-2792ec298503-kube-api-access-hpkhz\") pod \"authorino-7bcfd8f9cd-s47pl\" (UID: \"012c1b4f-5e26-49f2-a9b6-2792ec298503\") " pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl"
Sep 16 18:26:20.745696 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.745621    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddw98\" (UniqueName: \"kubernetes.io/projected/10431f42-a266-4912-bd1e-ff929d3f1059-kube-api-access-ddw98\") pod \"authorino-59c974bd48-gbw6z\" (UID: \"10431f42-a266-4912-bd1e-ff929d3f1059\") " pod="kuadrant-system/authorino-59c974bd48-gbw6z"
Sep 16 18:26:20.756022 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.755999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpkhz\" (UniqueName: \"kubernetes.io/projected/012c1b4f-5e26-49f2-a9b6-2792ec298503-kube-api-access-hpkhz\") pod \"authorino-7bcfd8f9cd-s47pl\" (UID: \"012c1b4f-5e26-49f2-a9b6-2792ec298503\") " pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl"
Sep 16 18:26:20.846084 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.846031    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddw98\" (UniqueName: \"kubernetes.io/projected/10431f42-a266-4912-bd1e-ff929d3f1059-kube-api-access-ddw98\") pod \"authorino-59c974bd48-gbw6z\" (UID: \"10431f42-a266-4912-bd1e-ff929d3f1059\") " pod="kuadrant-system/authorino-59c974bd48-gbw6z"
Sep 16 18:26:20.854874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.854850    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddw98\" (UniqueName: \"kubernetes.io/projected/10431f42-a266-4912-bd1e-ff929d3f1059-kube-api-access-ddw98\") pod \"authorino-59c974bd48-gbw6z\" (UID: \"10431f42-a266-4912-bd1e-ff929d3f1059\") " pod="kuadrant-system/authorino-59c974bd48-gbw6z"
Sep 16 18:26:20.859622 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.859607    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl"
Sep 16 18:26:20.968044 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:20.968017    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-gbw6z"
Sep 16 18:26:21.125014 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:21.124968    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-s47pl"]
Sep 16 18:26:21.127027 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:26:21.127001    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod012c1b4f_5e26_49f2_a9b6_2792ec298503.slice/crio-b700662057b22d5578d4ece376f96f9fe23d645a224cbddffe8059b17b16caf8 WatchSource:0}: Error finding container b700662057b22d5578d4ece376f96f9fe23d645a224cbddffe8059b17b16caf8: Status 404 returned error can't find the container with id b700662057b22d5578d4ece376f96f9fe23d645a224cbddffe8059b17b16caf8
Sep 16 18:26:21.233063 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:21.233043    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-gbw6z"]
Sep 16 18:26:21.234817 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:26:21.234797    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10431f42_a266_4912_bd1e_ff929d3f1059.slice/crio-bc7b15f3463dacc641f68b015cdd10933529d93c1439e691c2ca2b15d808fdd6 WatchSource:0}: Error finding container bc7b15f3463dacc641f68b015cdd10933529d93c1439e691c2ca2b15d808fdd6: Status 404 returned error can't find the container with id bc7b15f3463dacc641f68b015cdd10933529d93c1439e691c2ca2b15d808fdd6
Sep 16 18:26:21.814043 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:21.814018    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-gbw6z" event={"ID":"10431f42-a266-4912-bd1e-ff929d3f1059","Type":"ContainerStarted","Data":"bc7b15f3463dacc641f68b015cdd10933529d93c1439e691c2ca2b15d808fdd6"}
Sep 16 18:26:21.815093 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:21.815063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl" event={"ID":"012c1b4f-5e26-49f2-a9b6-2792ec298503","Type":"ContainerStarted","Data":"b700662057b22d5578d4ece376f96f9fe23d645a224cbddffe8059b17b16caf8"}
Sep 16 18:26:24.832002 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:24.831968    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl" event={"ID":"012c1b4f-5e26-49f2-a9b6-2792ec298503","Type":"ContainerStarted","Data":"30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517"}
Sep 16 18:26:24.833418 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:24.833395    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-gbw6z" event={"ID":"10431f42-a266-4912-bd1e-ff929d3f1059","Type":"ContainerStarted","Data":"dff5f96678d65fe72830562c6de3ba2d74f95145a1e85decbe727aa74aecaca4"}
Sep 16 18:26:24.846512 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:24.846472    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl" podStartSLOduration=2.07848234 podStartE2EDuration="4.846456983s" totalImagesPullingTime="2.767974643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:26:24.845648185 +0000 UTC m=+605.085436938" watchObservedRunningTime="2026-09-16 18:26:24.846456983 +0000 UTC m=+605.086245735"
Sep 16 18:26:24.861329 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:24.861288    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-gbw6z" podStartSLOduration=2.213057702 podStartE2EDuration="4.861274618s" totalImagesPullingTime="2.648216916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:26:24.859443561 +0000 UTC m=+605.099232313" watchObservedRunningTime="2026-09-16 18:26:24.861274618 +0000 UTC m=+605.101063370"
Sep 16 18:26:24.887902 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:24.887876    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-s47pl"]
Sep 16 18:26:26.841798 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:26.841757    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl" podUID="012c1b4f-5e26-49f2-a9b6-2792ec298503" containerName="authorino" containerID="cri-o://30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517" gracePeriod=30
Sep 16 18:26:27.079789 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.079765    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl"
Sep 16 18:26:27.194574 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.194552    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpkhz\" (UniqueName: \"kubernetes.io/projected/012c1b4f-5e26-49f2-a9b6-2792ec298503-kube-api-access-hpkhz\") pod \"012c1b4f-5e26-49f2-a9b6-2792ec298503\" (UID: \"012c1b4f-5e26-49f2-a9b6-2792ec298503\") "
Sep 16 18:26:27.196704 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.196681    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/012c1b4f-5e26-49f2-a9b6-2792ec298503-kube-api-access-hpkhz" (OuterVolumeSpecName: "kube-api-access-hpkhz") pod "012c1b4f-5e26-49f2-a9b6-2792ec298503" (UID: "012c1b4f-5e26-49f2-a9b6-2792ec298503"). InnerVolumeSpecName "kube-api-access-hpkhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:26:27.295336 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.295313    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpkhz\" (UniqueName: \"kubernetes.io/projected/012c1b4f-5e26-49f2-a9b6-2792ec298503-kube-api-access-hpkhz\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:26:27.846871 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.846846    2562 generic.go:356] "Generic (PLEG): container finished" podID="012c1b4f-5e26-49f2-a9b6-2792ec298503" containerID="30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517" exitCode=0
Sep 16 18:26:27.847239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.846883    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl"
Sep 16 18:26:27.847239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.846884    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl" event={"ID":"012c1b4f-5e26-49f2-a9b6-2792ec298503","Type":"ContainerDied","Data":"30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517"}
Sep 16 18:26:27.847239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.846924    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-s47pl" event={"ID":"012c1b4f-5e26-49f2-a9b6-2792ec298503","Type":"ContainerDied","Data":"b700662057b22d5578d4ece376f96f9fe23d645a224cbddffe8059b17b16caf8"}
Sep 16 18:26:27.847239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.846946    2562 scope.go:117] "RemoveContainer" containerID="30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517"
Sep 16 18:26:27.855352 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.855337    2562 scope.go:117] "RemoveContainer" containerID="30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517"
Sep 16 18:26:27.855610 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:26:27.855574    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517\": container with ID starting with 30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517 not found: ID does not exist" containerID="30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517"
Sep 16 18:26:27.855662 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.855622    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517"} err="failed to get container status \"30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517\": rpc error: code = NotFound desc = could not find container \"30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517\": container with ID starting with 30fcf9067b29fbcdda69e0f0e4dd7e3db580d7f0c1408664ecb1dbc152934517 not found: ID does not exist"
Sep 16 18:26:27.867664 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.867640    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-s47pl"]
Sep 16 18:26:27.871525 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:27.871507    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-s47pl"]
Sep 16 18:26:28.284801 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:28.284778    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="012c1b4f-5e26-49f2-a9b6-2792ec298503" path="/var/lib/kubelet/pods/012c1b4f-5e26-49f2-a9b6-2792ec298503/volumes"
Sep 16 18:26:36.813780 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:36.813721    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:36.922954 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:36.922913    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:37.291794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:37.291768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:48.965157 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:48.965064    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:49.186492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.186462    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-pprpf"]
Sep 16 18:26:49.186863 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.186846    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="012c1b4f-5e26-49f2-a9b6-2792ec298503" containerName="authorino"
Sep 16 18:26:49.186943 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.186865    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="012c1b4f-5e26-49f2-a9b6-2792ec298503" containerName="authorino"
Sep 16 18:26:49.187007 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.186954    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="012c1b4f-5e26-49f2-a9b6-2792ec298503" containerName="authorino"
Sep 16 18:26:49.189805 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.189787    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pprpf"
Sep 16 18:26:49.198063 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.198040    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-pprpf"]
Sep 16 18:26:49.249518 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.249454    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsllk\" (UniqueName: \"kubernetes.io/projected/2fdf8af0-684a-491c-85c9-01e566c2304d-kube-api-access-zsllk\") pod \"authorino-557f876f69-pprpf\" (UID: \"2fdf8af0-684a-491c-85c9-01e566c2304d\") " pod="kuadrant-system/authorino-557f876f69-pprpf"
Sep 16 18:26:49.318467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.318441    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:49.350371 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.350346    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsllk\" (UniqueName: \"kubernetes.io/projected/2fdf8af0-684a-491c-85c9-01e566c2304d-kube-api-access-zsllk\") pod \"authorino-557f876f69-pprpf\" (UID: \"2fdf8af0-684a-491c-85c9-01e566c2304d\") " pod="kuadrant-system/authorino-557f876f69-pprpf"
Sep 16 18:26:49.362212 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.362191    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsllk\" (UniqueName: \"kubernetes.io/projected/2fdf8af0-684a-491c-85c9-01e566c2304d-kube-api-access-zsllk\") pod \"authorino-557f876f69-pprpf\" (UID: \"2fdf8af0-684a-491c-85c9-01e566c2304d\") " pod="kuadrant-system/authorino-557f876f69-pprpf"
Sep 16 18:26:49.419758 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.419729    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:49.426628 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.426607    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-pprpf"]
Sep 16 18:26:49.426761 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.426751    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pprpf"
Sep 16 18:26:49.452052 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.452031    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bd46888f6-s94qx"]
Sep 16 18:26:49.455536 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.455521    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bd46888f6-s94qx"
Sep 16 18:26:49.468362 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.468301    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bd46888f6-s94qx"]
Sep 16 18:26:49.489203 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.489167    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bd46888f6-s94qx"]
Sep 16 18:26:49.489489 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:26:49.489462    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-x8wz8], unattached volumes=[], failed to process volumes=[kube-api-access-x8wz8]: context canceled" pod="kuadrant-system/authorino-5bd46888f6-s94qx" podUID="908be5b4-983a-46ba-84bd-e8d8abd367c6"
Sep 16 18:26:49.551957 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.551901    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8wz8\" (UniqueName: \"kubernetes.io/projected/908be5b4-983a-46ba-84bd-e8d8abd367c6-kube-api-access-x8wz8\") pod \"authorino-5bd46888f6-s94qx\" (UID: \"908be5b4-983a-46ba-84bd-e8d8abd367c6\") " pod="kuadrant-system/authorino-5bd46888f6-s94qx"
Sep 16 18:26:49.653260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.653236    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8wz8\" (UniqueName: \"kubernetes.io/projected/908be5b4-983a-46ba-84bd-e8d8abd367c6-kube-api-access-x8wz8\") pod \"authorino-5bd46888f6-s94qx\" (UID: \"908be5b4-983a-46ba-84bd-e8d8abd367c6\") " pod="kuadrant-system/authorino-5bd46888f6-s94qx"
Sep 16 18:26:49.662945 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.662926    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8wz8\" (UniqueName: \"kubernetes.io/projected/908be5b4-983a-46ba-84bd-e8d8abd367c6-kube-api-access-x8wz8\") pod \"authorino-5bd46888f6-s94qx\" (UID: \"908be5b4-983a-46ba-84bd-e8d8abd367c6\") " pod="kuadrant-system/authorino-5bd46888f6-s94qx"
Sep 16 18:26:49.702175 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.702156    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-pprpf"]
Sep 16 18:26:49.704814 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:26:49.704795    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fdf8af0_684a_491c_85c9_01e566c2304d.slice/crio-f523f64290bdef11108e3c829ef54f2b816b578c5ceab72addbd720ac1578ac5 WatchSource:0}: Error finding container f523f64290bdef11108e3c829ef54f2b816b578c5ceab72addbd720ac1578ac5: Status 404 returned error can't find the container with id f523f64290bdef11108e3c829ef54f2b816b578c5ceab72addbd720ac1578ac5
Sep 16 18:26:49.938275 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.938239    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-pprpf" event={"ID":"2fdf8af0-684a-491c-85c9-01e566c2304d","Type":"ContainerStarted","Data":"f523f64290bdef11108e3c829ef54f2b816b578c5ceab72addbd720ac1578ac5"}
Sep 16 18:26:49.938418 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.938305    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bd46888f6-s94qx"
Sep 16 18:26:49.945064 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:49.945048    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bd46888f6-s94qx"
Sep 16 18:26:50.056402 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:50.056384    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8wz8\" (UniqueName: \"kubernetes.io/projected/908be5b4-983a-46ba-84bd-e8d8abd367c6-kube-api-access-x8wz8\") pod \"908be5b4-983a-46ba-84bd-e8d8abd367c6\" (UID: \"908be5b4-983a-46ba-84bd-e8d8abd367c6\") "
Sep 16 18:26:50.058450 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:50.058426    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/908be5b4-983a-46ba-84bd-e8d8abd367c6-kube-api-access-x8wz8" (OuterVolumeSpecName: "kube-api-access-x8wz8") pod "908be5b4-983a-46ba-84bd-e8d8abd367c6" (UID: "908be5b4-983a-46ba-84bd-e8d8abd367c6"). InnerVolumeSpecName "kube-api-access-x8wz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:26:50.157029 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:50.157008    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8wz8\" (UniqueName: \"kubernetes.io/projected/908be5b4-983a-46ba-84bd-e8d8abd367c6-kube-api-access-x8wz8\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:26:50.944310 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:50.944276    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bd46888f6-s94qx"
Sep 16 18:26:50.944462 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:50.944326    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-pprpf" event={"ID":"2fdf8af0-684a-491c-85c9-01e566c2304d","Type":"ContainerStarted","Data":"a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb"}
Sep 16 18:26:50.944462 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:50.944333    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-pprpf" podUID="2fdf8af0-684a-491c-85c9-01e566c2304d" containerName="authorino" containerID="cri-o://a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb" gracePeriod=30
Sep 16 18:26:50.959302 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:50.959265    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-pprpf" podStartSLOduration=1.470970044 podStartE2EDuration="1.959251459s" totalImagesPullingTime="488.281415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:26:50.95805656 +0000 UTC m=+631.197845313" watchObservedRunningTime="2026-09-16 18:26:50.959251459 +0000 UTC m=+631.199040211"
Sep 16 18:26:50.984150 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:50.984129    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bd46888f6-s94qx"]
Sep 16 18:26:50.988190 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:50.988172    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5bd46888f6-s94qx"]
Sep 16 18:26:51.204170 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.204118    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pprpf"
Sep 16 18:26:51.266682 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.266660    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsllk\" (UniqueName: \"kubernetes.io/projected/2fdf8af0-684a-491c-85c9-01e566c2304d-kube-api-access-zsllk\") pod \"2fdf8af0-684a-491c-85c9-01e566c2304d\" (UID: \"2fdf8af0-684a-491c-85c9-01e566c2304d\") "
Sep 16 18:26:51.268801 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.268777    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fdf8af0-684a-491c-85c9-01e566c2304d-kube-api-access-zsllk" (OuterVolumeSpecName: "kube-api-access-zsllk") pod "2fdf8af0-684a-491c-85c9-01e566c2304d" (UID: "2fdf8af0-684a-491c-85c9-01e566c2304d"). InnerVolumeSpecName "kube-api-access-zsllk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:26:51.367785 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.367765    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsllk\" (UniqueName: \"kubernetes.io/projected/2fdf8af0-684a-491c-85c9-01e566c2304d-kube-api-access-zsllk\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:26:51.949271 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.949242    2562 generic.go:356] "Generic (PLEG): container finished" podID="2fdf8af0-684a-491c-85c9-01e566c2304d" containerID="a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb" exitCode=0
Sep 16 18:26:51.949385 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.949280    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pprpf"
Sep 16 18:26:51.949385 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.949327    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-pprpf" event={"ID":"2fdf8af0-684a-491c-85c9-01e566c2304d","Type":"ContainerDied","Data":"a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb"}
Sep 16 18:26:51.949385 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.949367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-pprpf" event={"ID":"2fdf8af0-684a-491c-85c9-01e566c2304d","Type":"ContainerDied","Data":"f523f64290bdef11108e3c829ef54f2b816b578c5ceab72addbd720ac1578ac5"}
Sep 16 18:26:51.949496 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.949385    2562 scope.go:117] "RemoveContainer" containerID="a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb"
Sep 16 18:26:51.957659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.957643    2562 scope.go:117] "RemoveContainer" containerID="a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb"
Sep 16 18:26:51.957905 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:26:51.957885    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb\": container with ID starting with a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb not found: ID does not exist" containerID="a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb"
Sep 16 18:26:51.957967 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.957914    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb"} err="failed to get container status \"a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb\": rpc error: code = NotFound desc = could not find container \"a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb\": container with ID starting with a3e6098a2927a706d5b76e0f61274fd13c7fe0267e1252ea06d3f90fe0f2dcdb not found: ID does not exist"
Sep 16 18:26:51.968114 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.968092    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-pprpf"]
Sep 16 18:26:51.971731 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:51.971711    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-pprpf"]
Sep 16 18:26:52.284469 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:52.284422    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fdf8af0-684a-491c-85c9-01e566c2304d" path="/var/lib/kubelet/pods/2fdf8af0-684a-491c-85c9-01e566c2304d/volumes"
Sep 16 18:26:52.284758 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:52.284715    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="908be5b4-983a-46ba-84bd-e8d8abd367c6" path="/var/lib/kubelet/pods/908be5b4-983a-46ba-84bd-e8d8abd367c6/volumes"
Sep 16 18:26:54.730833 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:54.730805    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-gbw6z"]
Sep 16 18:26:54.733127 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:54.730948    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-gbw6z" podUID="10431f42-a266-4912-bd1e-ff929d3f1059" containerName="authorino" containerID="cri-o://dff5f96678d65fe72830562c6de3ba2d74f95145a1e85decbe727aa74aecaca4" gracePeriod=30
Sep 16 18:26:54.969452 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:54.969420    2562 generic.go:356] "Generic (PLEG): container finished" podID="10431f42-a266-4912-bd1e-ff929d3f1059" containerID="dff5f96678d65fe72830562c6de3ba2d74f95145a1e85decbe727aa74aecaca4" exitCode=0
Sep 16 18:26:54.969554 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:54.969470    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-gbw6z" event={"ID":"10431f42-a266-4912-bd1e-ff929d3f1059","Type":"ContainerDied","Data":"dff5f96678d65fe72830562c6de3ba2d74f95145a1e85decbe727aa74aecaca4"}
Sep 16 18:26:54.969554 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:54.969496    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-gbw6z" event={"ID":"10431f42-a266-4912-bd1e-ff929d3f1059","Type":"ContainerDied","Data":"bc7b15f3463dacc641f68b015cdd10933529d93c1439e691c2ca2b15d808fdd6"}
Sep 16 18:26:54.969554 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:54.969507    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc7b15f3463dacc641f68b015cdd10933529d93c1439e691c2ca2b15d808fdd6"
Sep 16 18:26:54.976416 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:54.976395    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-gbw6z"
Sep 16 18:26:55.097362 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:55.097310    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddw98\" (UniqueName: \"kubernetes.io/projected/10431f42-a266-4912-bd1e-ff929d3f1059-kube-api-access-ddw98\") pod \"10431f42-a266-4912-bd1e-ff929d3f1059\" (UID: \"10431f42-a266-4912-bd1e-ff929d3f1059\") "
Sep 16 18:26:55.099291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:55.099271    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10431f42-a266-4912-bd1e-ff929d3f1059-kube-api-access-ddw98" (OuterVolumeSpecName: "kube-api-access-ddw98") pod "10431f42-a266-4912-bd1e-ff929d3f1059" (UID: "10431f42-a266-4912-bd1e-ff929d3f1059"). InnerVolumeSpecName "kube-api-access-ddw98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:26:55.198490 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:55.198467    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddw98\" (UniqueName: \"kubernetes.io/projected/10431f42-a266-4912-bd1e-ff929d3f1059-kube-api-access-ddw98\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:26:55.973845 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:55.973819    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-gbw6z"
Sep 16 18:26:55.992424 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:55.992396    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-gbw6z"]
Sep 16 18:26:55.998280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:55.998256    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-gbw6z"]
Sep 16 18:26:56.284968 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:56.284950    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10431f42-a266-4912-bd1e-ff929d3f1059" path="/var/lib/kubelet/pods/10431f42-a266-4912-bd1e-ff929d3f1059/volumes"
Sep 16 18:26:59.387505 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.387460    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:26:59.402185 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.402163    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85f89b5f46-rjm7g"]
Sep 16 18:26:59.402512 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.402500    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10431f42-a266-4912-bd1e-ff929d3f1059" containerName="authorino"
Sep 16 18:26:59.402562 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.402513    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="10431f42-a266-4912-bd1e-ff929d3f1059" containerName="authorino"
Sep 16 18:26:59.402562 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.402527    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fdf8af0-684a-491c-85c9-01e566c2304d" containerName="authorino"
Sep 16 18:26:59.402562 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.402533    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fdf8af0-684a-491c-85c9-01e566c2304d" containerName="authorino"
Sep 16 18:26:59.402691 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.402581    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fdf8af0-684a-491c-85c9-01e566c2304d" containerName="authorino"
Sep 16 18:26:59.402691 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.402602    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="10431f42-a266-4912-bd1e-ff929d3f1059" containerName="authorino"
Sep 16 18:26:59.406449 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.406425    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:26:59.408887 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.408868    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 18:26:59.408995 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.408910    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 18:26:59.408995 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.408968    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 18:26:59.409090 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.408998    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qfs4j\""
Sep 16 18:26:59.415796 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.415777    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85f89b5f46-rjm7g"]
Sep 16 18:26:59.529254 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.529227    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-webhook-cert\") pod \"maas-controller-85f89b5f46-rjm7g\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") " pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:26:59.529356 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.529290    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4bx9\" (UniqueName: \"kubernetes.io/projected/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-kube-api-access-q4bx9\") pod \"maas-controller-85f89b5f46-rjm7g\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") " pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:26:59.529409 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.529363    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs\") pod \"maas-controller-85f89b5f46-rjm7g\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") " pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:26:59.630330 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.630307    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4bx9\" (UniqueName: \"kubernetes.io/projected/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-kube-api-access-q4bx9\") pod \"maas-controller-85f89b5f46-rjm7g\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") " pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:26:59.630417 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.630340    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs\") pod \"maas-controller-85f89b5f46-rjm7g\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") " pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:26:59.630417 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.630365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-webhook-cert\") pod \"maas-controller-85f89b5f46-rjm7g\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") " pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:26:59.630513 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:26:59.630452    2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 16 18:26:59.630513 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:26:59.630508    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs podName:e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:00.130492337 +0000 UTC m=+640.370281068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs") pod "maas-controller-85f89b5f46-rjm7g" (UID: "e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02") : secret "maas-controller-metrics-tls" not found
Sep 16 18:26:59.632719 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.632698    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-webhook-cert\") pod \"maas-controller-85f89b5f46-rjm7g\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") " pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:26:59.638528 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:26:59.638466    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4bx9\" (UniqueName: \"kubernetes.io/projected/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-kube-api-access-q4bx9\") pod \"maas-controller-85f89b5f46-rjm7g\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") " pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:27:00.135298 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:00.135272    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs\") pod \"maas-controller-85f89b5f46-rjm7g\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") " pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:27:00.135403 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:27:00.135391    2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 16 18:27:00.135445 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:27:00.135438    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs podName:e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:01.135422174 +0000 UTC m=+641.375210905 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs") pod "maas-controller-85f89b5f46-rjm7g" (UID: "e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02") : secret "maas-controller-metrics-tls" not found
Sep 16 18:27:00.802229 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:00.802200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:01.141916 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:01.141840    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs\") pod \"maas-controller-85f89b5f46-rjm7g\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") " pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:27:01.144239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:01.144219    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs\") pod \"maas-controller-85f89b5f46-rjm7g\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") " pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:27:01.217389 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:01.217358    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:27:01.487199 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:01.487177    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85f89b5f46-rjm7g"]
Sep 16 18:27:01.489656 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:27:01.489623    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode32d6f1e_6fe6_46b2_ad8d_0cdcdb319c02.slice/crio-671b61a77d9ee2baf482767440cd27c59e562e7d994440bcb47b1eac6823980f WatchSource:0}: Error finding container 671b61a77d9ee2baf482767440cd27c59e562e7d994440bcb47b1eac6823980f: Status 404 returned error can't find the container with id 671b61a77d9ee2baf482767440cd27c59e562e7d994440bcb47b1eac6823980f
Sep 16 18:27:02.000101 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:02.000075    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85f89b5f46-rjm7g" event={"ID":"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02","Type":"ContainerStarted","Data":"671b61a77d9ee2baf482767440cd27c59e562e7d994440bcb47b1eac6823980f"}
Sep 16 18:27:03.954190 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:03.954157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:04.010694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:04.010664    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85f89b5f46-rjm7g" event={"ID":"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02","Type":"ContainerStarted","Data":"2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da"}
Sep 16 18:27:04.010844 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:04.010766    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:27:04.031288 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:04.031244    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85f89b5f46-rjm7g" podStartSLOduration=2.7202869400000003 podStartE2EDuration="5.031232098s" totalImagesPullingTime="2.310945158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:27:04.029103219 +0000 UTC m=+644.268891985" watchObservedRunningTime="2026-09-16 18:27:04.031232098 +0000 UTC m=+644.271020849"
Sep 16 18:27:04.210636 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:04.210573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:15.016556 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:15.016529    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:27:15.772133 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:15.772098    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85f89b5f46-rjm7g"]
Sep 16 18:27:15.772307 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:15.772284    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-85f89b5f46-rjm7g" podUID="e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02" containerName="manager" containerID="cri-o://2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da" gracePeriod=30
Sep 16 18:27:16.010552 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.010532    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:27:16.064599 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.064520    2562 generic.go:356] "Generic (PLEG): container finished" podID="e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02" containerID="2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da" exitCode=0
Sep 16 18:27:16.064599 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.064558    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85f89b5f46-rjm7g" event={"ID":"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02","Type":"ContainerDied","Data":"2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da"}
Sep 16 18:27:16.064961 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.064612    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85f89b5f46-rjm7g" event={"ID":"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02","Type":"ContainerDied","Data":"671b61a77d9ee2baf482767440cd27c59e562e7d994440bcb47b1eac6823980f"}
Sep 16 18:27:16.064961 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.064558    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85f89b5f46-rjm7g"
Sep 16 18:27:16.064961 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.064631    2562 scope.go:117] "RemoveContainer" containerID="2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da"
Sep 16 18:27:16.072966 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.072948    2562 scope.go:117] "RemoveContainer" containerID="2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da"
Sep 16 18:27:16.073230 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:27:16.073212    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da\": container with ID starting with 2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da not found: ID does not exist" containerID="2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da"
Sep 16 18:27:16.073284 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.073239    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da"} err="failed to get container status \"2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da\": rpc error: code = NotFound desc = could not find container \"2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da\": container with ID starting with 2b4d0eefce79dc897be882ed2fa298ebe1020f2543a632801a59495f6fb130da not found: ID does not exist"
Sep 16 18:27:16.154524 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.154505    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs\") pod \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") "
Sep 16 18:27:16.154631 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.154531    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4bx9\" (UniqueName: \"kubernetes.io/projected/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-kube-api-access-q4bx9\") pod \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") "
Sep 16 18:27:16.154631 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.154549    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-webhook-cert\") pod \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\" (UID: \"e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02\") "
Sep 16 18:27:16.156746 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.156719    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02" (UID: "e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:27:16.156827 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.156749    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02" (UID: "e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:27:16.156827 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.156754    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-kube-api-access-q4bx9" (OuterVolumeSpecName: "kube-api-access-q4bx9") pod "e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02" (UID: "e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02"). InnerVolumeSpecName "kube-api-access-q4bx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:27:16.255459 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.255430    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-metrics-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:27:16.255459 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.255456    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4bx9\" (UniqueName: \"kubernetes.io/projected/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-kube-api-access-q4bx9\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:27:16.255459 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.255466    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02-webhook-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:27:16.380577 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.380519    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85f89b5f46-rjm7g"]
Sep 16 18:27:16.383560 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:16.383542    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-85f89b5f46-rjm7g"]
Sep 16 18:27:18.285489 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:18.285461    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02" path="/var/lib/kubelet/pods/e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02/volumes"
Sep 16 18:27:21.340927 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:21.340892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:21.405174 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:21.405149    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:35.589785 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:35.589747    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:35.693686 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:35.693657    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:36.991994 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:36.991956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:37.091408 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.091374    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:37.162193 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.162163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:37.293946 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.293874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:37.470606 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.470559    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:37.568837 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.568761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:37.583611 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.583559    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"]
Sep 16 18:27:37.583948 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.583932    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02" containerName="manager"
Sep 16 18:27:37.583948 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.583949    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02" containerName="manager"
Sep 16 18:27:37.584042 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.584029    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e32d6f1e-6fe6-46b2-ad8d-0cdcdb319c02" containerName="manager"
Sep 16 18:27:37.588357 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.588343    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"
Sep 16 18:27:37.590940 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.590914    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8jxlj\""
Sep 16 18:27:37.590940 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.590931    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 18:27:37.595709 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.595687    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"]
Sep 16 18:27:37.601073 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.601054    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"]
Sep 16 18:27:37.603093 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.603078    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"
Sep 16 18:27:37.604179 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.604159    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhm72\" (UniqueName: \"kubernetes.io/projected/dd0d133b-b605-4231-9399-72b5db35bb51-kube-api-access-bhm72\") pod \"payload-pre-processing-59fbc47bf8-ss6mc\" (UID: \"dd0d133b-b605-4231-9399-72b5db35bb51\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"
Sep 16 18:27:37.604267 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.604189    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0d133b-b605-4231-9399-72b5db35bb51-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ss6mc\" (UID: \"dd0d133b-b605-4231-9399-72b5db35bb51\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"
Sep 16 18:27:37.614979 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.614954    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"]
Sep 16 18:27:37.704861 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.704836    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1963517c-bd16-44cd-9994-8a432db84f92-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-pjwhq\" (UID: \"1963517c-bd16-44cd-9994-8a432db84f92\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"
Sep 16 18:27:37.704978 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.704911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2bgd\" (UniqueName: \"kubernetes.io/projected/1963517c-bd16-44cd-9994-8a432db84f92-kube-api-access-b2bgd\") pod \"payload-processing-5b54b7fbb7-pjwhq\" (UID: \"1963517c-bd16-44cd-9994-8a432db84f92\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"
Sep 16 18:27:37.704978 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.704938    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhm72\" (UniqueName: \"kubernetes.io/projected/dd0d133b-b605-4231-9399-72b5db35bb51-kube-api-access-bhm72\") pod \"payload-pre-processing-59fbc47bf8-ss6mc\" (UID: \"dd0d133b-b605-4231-9399-72b5db35bb51\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"
Sep 16 18:27:37.704978 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.704961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0d133b-b605-4231-9399-72b5db35bb51-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ss6mc\" (UID: \"dd0d133b-b605-4231-9399-72b5db35bb51\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"
Sep 16 18:27:37.705658 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.705635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd0d133b-b605-4231-9399-72b5db35bb51-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ss6mc\" (UID: \"dd0d133b-b605-4231-9399-72b5db35bb51\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"
Sep 16 18:27:37.715765 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.715743    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhm72\" (UniqueName: \"kubernetes.io/projected/dd0d133b-b605-4231-9399-72b5db35bb51-kube-api-access-bhm72\") pod \"payload-pre-processing-59fbc47bf8-ss6mc\" (UID: \"dd0d133b-b605-4231-9399-72b5db35bb51\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"
Sep 16 18:27:37.805779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.805760    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bgd\" (UniqueName: \"kubernetes.io/projected/1963517c-bd16-44cd-9994-8a432db84f92-kube-api-access-b2bgd\") pod \"payload-processing-5b54b7fbb7-pjwhq\" (UID: \"1963517c-bd16-44cd-9994-8a432db84f92\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"
Sep 16 18:27:37.805874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.805798    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1963517c-bd16-44cd-9994-8a432db84f92-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-pjwhq\" (UID: \"1963517c-bd16-44cd-9994-8a432db84f92\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"
Sep 16 18:27:37.806323 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.806308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1963517c-bd16-44cd-9994-8a432db84f92-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-pjwhq\" (UID: \"1963517c-bd16-44cd-9994-8a432db84f92\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"
Sep 16 18:27:37.813295 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.813270    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2bgd\" (UniqueName: \"kubernetes.io/projected/1963517c-bd16-44cd-9994-8a432db84f92-kube-api-access-b2bgd\") pod \"payload-processing-5b54b7fbb7-pjwhq\" (UID: \"1963517c-bd16-44cd-9994-8a432db84f92\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"
Sep 16 18:27:37.898574 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.898522    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"
Sep 16 18:27:37.914661 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:37.914642    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"
Sep 16 18:27:38.169762 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:38.169741    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"]
Sep 16 18:27:38.171477 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:27:38.171448    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd0d133b_b605_4231_9399_72b5db35bb51.slice/crio-e5fe5c07e7f6a2fdd3816c58fe8d673a26193750c4cad86d28279429d45307f7 WatchSource:0}: Error finding container e5fe5c07e7f6a2fdd3816c58fe8d673a26193750c4cad86d28279429d45307f7: Status 404 returned error can't find the container with id e5fe5c07e7f6a2fdd3816c58fe8d673a26193750c4cad86d28279429d45307f7
Sep 16 18:27:38.173325 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:38.173300    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:27:38.196988 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:38.196965    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"]
Sep 16 18:27:38.199385 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:27:38.199364    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1963517c_bd16_44cd_9994_8a432db84f92.slice/crio-e8e3f874478b398ffdb0e35cdae9d8a0d509c8dd6db083ea6b3ca1a0fc9e2d17 WatchSource:0}: Error finding container e8e3f874478b398ffdb0e35cdae9d8a0d509c8dd6db083ea6b3ca1a0fc9e2d17: Status 404 returned error can't find the container with id e8e3f874478b398ffdb0e35cdae9d8a0d509c8dd6db083ea6b3ca1a0fc9e2d17
Sep 16 18:27:39.163283 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:39.163246    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc" event={"ID":"dd0d133b-b605-4231-9399-72b5db35bb51","Type":"ContainerStarted","Data":"e5fe5c07e7f6a2fdd3816c58fe8d673a26193750c4cad86d28279429d45307f7"}
Sep 16 18:27:39.164658 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:39.164630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq" event={"ID":"1963517c-bd16-44cd-9994-8a432db84f92","Type":"ContainerStarted","Data":"e8e3f874478b398ffdb0e35cdae9d8a0d509c8dd6db083ea6b3ca1a0fc9e2d17"}
Sep 16 18:27:40.170794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:40.170752    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq" event={"ID":"1963517c-bd16-44cd-9994-8a432db84f92","Type":"ContainerStarted","Data":"64be010291b20058f3a8e6c2ded27dcade4dc95703cc6edf47adb925e64286f5"}
Sep 16 18:27:40.171290 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:40.170850    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"
Sep 16 18:27:40.172285 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:40.172266    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc" event={"ID":"dd0d133b-b605-4231-9399-72b5db35bb51","Type":"ContainerStarted","Data":"0e9344f364801fce34e59b73f09356b6143eac57afff063b83db522841f23ed0"}
Sep 16 18:27:40.172373 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:40.172335    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"
Sep 16 18:27:40.189444 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:40.189403    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq" podStartSLOduration=2.015069281 podStartE2EDuration="3.189390939s" totalImagesPullingTime="1.174321658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:27:40.1878492 +0000 UTC m=+680.427637956" watchObservedRunningTime="2026-09-16 18:27:40.189390939 +0000 UTC m=+680.429179691"
Sep 16 18:27:40.204063 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:40.204020    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc" podStartSLOduration=1.999594973 podStartE2EDuration="3.204009419s" totalImagesPullingTime="1.204414446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:27:40.203557488 +0000 UTC m=+680.443346240" watchObservedRunningTime="2026-09-16 18:27:40.204009419 +0000 UTC m=+680.443798172"
Sep 16 18:27:47.539908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:47.539880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:47.634575 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:47.634542    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:51.179024 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:51.178997    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ss6mc"
Sep 16 18:27:51.179465 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:51.179120    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-pjwhq"
Sep 16 18:27:54.687672 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.687639    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"]
Sep 16 18:27:54.689838 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.689820    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.692423 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.692402    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9ct5h\""
Sep 16 18:27:54.692642 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.692620    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 16 18:27:54.692783 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.692668    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.692783 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.692719    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.701850 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.701830    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"]
Sep 16 18:27:54.724313 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.724280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:27:54.730292 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.730273    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.730354 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.730301    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkfm4\" (UniqueName: \"kubernetes.io/projected/3d5b4998-680b-45c0-a4ad-470066799f65-kube-api-access-rkfm4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.730354 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.730332    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.730354 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.730345    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.730453 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.730360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d5b4998-680b-45c0-a4ad-470066799f65-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.730453 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.730385    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.831113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.831089    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.831113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.831114    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkfm4\" (UniqueName: \"kubernetes.io/projected/3d5b4998-680b-45c0-a4ad-470066799f65-kube-api-access-rkfm4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.831268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.831140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.831268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.831153    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.831268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.831168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d5b4998-680b-45c0-a4ad-470066799f65-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.831268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.831189    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.831569 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.831543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.831569 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.831559    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.831693 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.831637    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.833416 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.833401    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d5b4998-680b-45c0-a4ad-470066799f65-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.833795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.833777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d5b4998-680b-45c0-a4ad-470066799f65-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:54.839856 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:54.839834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkfm4\" (UniqueName: \"kubernetes.io/projected/3d5b4998-680b-45c0-a4ad-470066799f65-kube-api-access-rkfm4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv\" (UID: \"3d5b4998-680b-45c0-a4ad-470066799f65\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:55.002204 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:55.002152    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:27:55.271364 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:55.271342    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"]
Sep 16 18:27:55.273334 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:27:55.273308    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d5b4998_680b_45c0_a4ad_470066799f65.slice/crio-058bd25fe4b4607c61b484bbe093f6c93365e31cef4896781110db80f3fd7d5d WatchSource:0}: Error finding container 058bd25fe4b4607c61b484bbe093f6c93365e31cef4896781110db80f3fd7d5d: Status 404 returned error can't find the container with id 058bd25fe4b4607c61b484bbe093f6c93365e31cef4896781110db80f3fd7d5d
Sep 16 18:27:56.242043 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:56.242020    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv" event={"ID":"3d5b4998-680b-45c0-a4ad-470066799f65","Type":"ContainerStarted","Data":"058bd25fe4b4607c61b484bbe093f6c93365e31cef4896781110db80f3fd7d5d"}
Sep 16 18:27:57.247744 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:57.247714    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv" event={"ID":"3d5b4998-680b-45c0-a4ad-470066799f65","Type":"ContainerStarted","Data":"a5a5f9cf702280a8c238c133caf666082d4739df6b57195a52ff446a6aebd8bc"}
Sep 16 18:27:57.268478 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:27:57.268426    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv" podStartSLOduration=2.349541134 podStartE2EDuration="3.268412543s" totalImagesPullingTime="918.871409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:27:57.266898577 +0000 UTC m=+697.506687330" watchObservedRunningTime="2026-09-16 18:27:57.268412543 +0000 UTC m=+697.508201294"
Sep 16 18:28:01.330092 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:01.330056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:05.002446 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:05.002410    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:28:05.002915 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:05.002465    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:28:05.014202 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:05.014184    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:28:05.291397 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:05.291342    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv"
Sep 16 18:28:25.338450 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:25.338419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:30.001177 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.001144    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"]
Sep 16 18:28:30.004925 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.004908    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.007444 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.007423    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 16 18:28:30.016741 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.016722    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"]
Sep 16 18:28:30.029368 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.029348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:30.091408 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.091385    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.091506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.091414    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.091506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.091450    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.091506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.091474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70cee47-467a-43ba-8a4a-ac183fa2d74e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.091705 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.091567    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sdv9\" (UniqueName: \"kubernetes.io/projected/f70cee47-467a-43ba-8a4a-ac183fa2d74e-kube-api-access-7sdv9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.091705 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.091645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.192373 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.192349    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.192455 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.192383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.192455 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.192425    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.192455 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.192449    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70cee47-467a-43ba-8a4a-ac183fa2d74e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.192629 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.192504    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sdv9\" (UniqueName: \"kubernetes.io/projected/f70cee47-467a-43ba-8a4a-ac183fa2d74e-kube-api-access-7sdv9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.192629 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.192536    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.192807 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.192787    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.192874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.192820    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.192874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.192837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.194703 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.194682    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70cee47-467a-43ba-8a4a-ac183fa2d74e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.194991 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.194965    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70cee47-467a-43ba-8a4a-ac183fa2d74e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.202702 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.202684    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sdv9\" (UniqueName: \"kubernetes.io/projected/f70cee47-467a-43ba-8a4a-ac183fa2d74e-kube-api-access-7sdv9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mwztf\" (UID: \"f70cee47-467a-43ba-8a4a-ac183fa2d74e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.314633 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.314578    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:30.588948 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:30.588929    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"]
Sep 16 18:28:30.590745 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:28:30.590720    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf70cee47_467a_43ba_8a4a_ac183fa2d74e.slice/crio-63529d18112e5c15d93f34d504a9b4deb914c9a3dc22054b8f2bc3798532b70a WatchSource:0}: Error finding container 63529d18112e5c15d93f34d504a9b4deb914c9a3dc22054b8f2bc3798532b70a: Status 404 returned error can't find the container with id 63529d18112e5c15d93f34d504a9b4deb914c9a3dc22054b8f2bc3798532b70a
Sep 16 18:28:31.391087 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:31.391059    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf" event={"ID":"f70cee47-467a-43ba-8a4a-ac183fa2d74e","Type":"ContainerStarted","Data":"066dfd8b818bfdf2dbeb3962189b2979ed9601941360517271fe9fba1b2c0cc5"}
Sep 16 18:28:31.391429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:31.391103    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf" event={"ID":"f70cee47-467a-43ba-8a4a-ac183fa2d74e","Type":"ContainerStarted","Data":"63529d18112e5c15d93f34d504a9b4deb914c9a3dc22054b8f2bc3798532b70a"}
Sep 16 18:28:31.411843 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:31.411791    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf" podStartSLOduration=2.196467527 podStartE2EDuration="2.411774857s" totalImagesPullingTime="215.30733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:31.411445634 +0000 UTC m=+731.651234386" watchObservedRunningTime="2026-09-16 18:28:31.411774857 +0000 UTC m=+731.651563610"
Sep 16 18:28:36.048724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:36.048689    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:40.315459 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:40.315430    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:40.315916 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:40.315473    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:40.327202 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:40.327180    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:40.438285 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:40.438264    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mwztf"
Sep 16 18:28:51.621036 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:51.620999    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:51.721366 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:51.721328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:57.186467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:28:57.186431    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:08.363478 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:08.363442    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:17.208250 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.208220    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"]
Sep 16 18:29:17.211579 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.211559    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.214841 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.214823    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 16 18:29:17.223619 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.223573    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"]
Sep 16 18:29:17.230529 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.230505    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.230681 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.230543    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.230681 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.230573    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.230681 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.230670    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.230866 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.230702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80594d49-9246-423f-8411-9a52f079d218-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.230866 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.230717    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kstf\" (UniqueName: \"kubernetes.io/projected/80594d49-9246-423f-8411-9a52f079d218-kube-api-access-4kstf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.265886 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.265861    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:17.331247 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.331230    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.331347 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.331261    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.331347 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.331281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.331347 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.331314    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.331347 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.331340    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80594d49-9246-423f-8411-9a52f079d218-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.331570 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.331358    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kstf\" (UniqueName: \"kubernetes.io/projected/80594d49-9246-423f-8411-9a52f079d218-kube-api-access-4kstf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.331780 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.331759    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.331841 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.331773    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.331841 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.331761    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.333448 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.333430    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80594d49-9246-423f-8411-9a52f079d218-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.333869 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.333848    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80594d49-9246-423f-8411-9a52f079d218-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.339746 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.339727    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kstf\" (UniqueName: \"kubernetes.io/projected/80594d49-9246-423f-8411-9a52f079d218-kube-api-access-4kstf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m\" (UID: \"80594d49-9246-423f-8411-9a52f079d218\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.521218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.521168    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:17.794258 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:17.794234    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"]
Sep 16 18:29:17.796062 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:29:17.796032    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80594d49_9246_423f_8411_9a52f079d218.slice/crio-71f9d163a48da9d1028370973638840488530ed75b6dec84385a107e3089e5d7 WatchSource:0}: Error finding container 71f9d163a48da9d1028370973638840488530ed75b6dec84385a107e3089e5d7: Status 404 returned error can't find the container with id 71f9d163a48da9d1028370973638840488530ed75b6dec84385a107e3089e5d7
Sep 16 18:29:18.598036 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:18.598002    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m" event={"ID":"80594d49-9246-423f-8411-9a52f079d218","Type":"ContainerStarted","Data":"4e2ebe8c5c6deb31d3f26ca239f62dc846b82fdfb400c6cd1e83851e0a39ddfc"}
Sep 16 18:29:18.598036 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:18.598039    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m" event={"ID":"80594d49-9246-423f-8411-9a52f079d218","Type":"ContainerStarted","Data":"71f9d163a48da9d1028370973638840488530ed75b6dec84385a107e3089e5d7"}
Sep 16 18:29:18.615484 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:18.615437    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m" podStartSLOduration=1.42165808 podStartE2EDuration="1.615422272s" totalImagesPullingTime="193.764192ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:18.614904817 +0000 UTC m=+778.854693568" watchObservedRunningTime="2026-09-16 18:29:18.615422272 +0000 UTC m=+778.855211023"
Sep 16 18:29:27.522127 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:27.522098    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:27.522554 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:27.522137    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:27.533954 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:27.533933    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:27.648840 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:27.648813    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m"
Sep 16 18:29:32.291471 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:29:32.291443    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:05.948766 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:05.948731    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f57666fc5-4mmlh"]
Sep 16 18:30:05.952245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:05.952223    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f57666fc5-4mmlh"
Sep 16 18:30:05.956524 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:05.956503    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 16 18:30:05.956524 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:05.956512    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vh4db\""
Sep 16 18:30:05.962914 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:05.962893    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f57666fc5-4mmlh"]
Sep 16 18:30:06.097453 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:06.097428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn2vt\" (UniqueName: \"kubernetes.io/projected/33c1c5fa-c340-4dc8-9478-0108e4eef38c-kube-api-access-zn2vt\") pod \"authorino-6f57666fc5-4mmlh\" (UID: \"33c1c5fa-c340-4dc8-9478-0108e4eef38c\") " pod="kuadrant-system/authorino-6f57666fc5-4mmlh"
Sep 16 18:30:06.097550 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:06.097481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33c1c5fa-c340-4dc8-9478-0108e4eef38c-tls-cert\") pod \"authorino-6f57666fc5-4mmlh\" (UID: \"33c1c5fa-c340-4dc8-9478-0108e4eef38c\") " pod="kuadrant-system/authorino-6f57666fc5-4mmlh"
Sep 16 18:30:06.198581 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:06.198556    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn2vt\" (UniqueName: \"kubernetes.io/projected/33c1c5fa-c340-4dc8-9478-0108e4eef38c-kube-api-access-zn2vt\") pod \"authorino-6f57666fc5-4mmlh\" (UID: \"33c1c5fa-c340-4dc8-9478-0108e4eef38c\") " pod="kuadrant-system/authorino-6f57666fc5-4mmlh"
Sep 16 18:30:06.198694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:06.198645    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33c1c5fa-c340-4dc8-9478-0108e4eef38c-tls-cert\") pod \"authorino-6f57666fc5-4mmlh\" (UID: \"33c1c5fa-c340-4dc8-9478-0108e4eef38c\") " pod="kuadrant-system/authorino-6f57666fc5-4mmlh"
Sep 16 18:30:06.201093 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:06.201050    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33c1c5fa-c340-4dc8-9478-0108e4eef38c-tls-cert\") pod \"authorino-6f57666fc5-4mmlh\" (UID: \"33c1c5fa-c340-4dc8-9478-0108e4eef38c\") " pod="kuadrant-system/authorino-6f57666fc5-4mmlh"
Sep 16 18:30:06.206662 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:06.206644    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn2vt\" (UniqueName: \"kubernetes.io/projected/33c1c5fa-c340-4dc8-9478-0108e4eef38c-kube-api-access-zn2vt\") pod \"authorino-6f57666fc5-4mmlh\" (UID: \"33c1c5fa-c340-4dc8-9478-0108e4eef38c\") " pod="kuadrant-system/authorino-6f57666fc5-4mmlh"
Sep 16 18:30:06.260669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:06.260647    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f57666fc5-4mmlh"
Sep 16 18:30:06.532345 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:06.532268    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f57666fc5-4mmlh"]
Sep 16 18:30:06.535340 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:30:06.535305    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33c1c5fa_c340_4dc8_9478_0108e4eef38c.slice/crio-8a3d3f02d17d3dcc4f96db48b5417716d1aa51fb5ae20f1251e5ed72575d65c5 WatchSource:0}: Error finding container 8a3d3f02d17d3dcc4f96db48b5417716d1aa51fb5ae20f1251e5ed72575d65c5: Status 404 returned error can't find the container with id 8a3d3f02d17d3dcc4f96db48b5417716d1aa51fb5ae20f1251e5ed72575d65c5
Sep 16 18:30:06.807087 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:06.807028    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f57666fc5-4mmlh" event={"ID":"33c1c5fa-c340-4dc8-9478-0108e4eef38c","Type":"ContainerStarted","Data":"8a3d3f02d17d3dcc4f96db48b5417716d1aa51fb5ae20f1251e5ed72575d65c5"}
Sep 16 18:30:07.816366 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:07.816321    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f57666fc5-4mmlh" event={"ID":"33c1c5fa-c340-4dc8-9478-0108e4eef38c","Type":"ContainerStarted","Data":"98d8b82da809fc146ae41744319320ae8681491bdeea4cf39fbd99336557f12d"}
Sep 16 18:30:07.837689 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:07.837634    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6f57666fc5-4mmlh" podStartSLOduration=2.356988486 podStartE2EDuration="2.837616092s" totalImagesPullingTime="480.627606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:07.836805766 +0000 UTC m=+828.076594531" watchObservedRunningTime="2026-09-16 18:30:07.837616092 +0000 UTC m=+828.077404845"
Sep 16 18:30:08.416002 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:08.415968    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:08.512320 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:08.512274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:09.597481 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:09.597443    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:09.698409 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:09.698377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:28.451929 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:28.451893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:28.553487 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:28.553458    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:28.653740 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:28.653712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:28.916149 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:28.916102    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:29.046185 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:29.046149    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:29.147555 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:29.147528    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:29.239623 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:29.239534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:29.484454 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:29.484426    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:29.734182 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:29.734158    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:29.982331 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:29.982304    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:30.083987 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:30.083910    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:30.435950 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:30.435926    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:31.817965 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:31.817934    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:31.982754 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:31.982712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:32.080553 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:32.080478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:32.972854 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:32.972820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:33.072515 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:33.072477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:33.704016 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:33.703989    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:33.765754 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:33.765726    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:34.054439 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.054370    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:34.410559 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.410504    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:34.503851 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.503828    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"]
Sep 16 18:30:34.505793 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.505766    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:34.507183 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.507168    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:30:34.509547 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.509525    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-0197e4\""
Sep 16 18:30:34.509673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.509583    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-0197e4-dockercfg-kmm82\""
Sep 16 18:30:34.518279 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.518257    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"]
Sep 16 18:30:34.522975 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.522955    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9675c1ec-78db-4511-8621-ff9ab01f822a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27\" (UID: \"9675c1ec-78db-4511-8621-ff9ab01f822a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:30:34.523078 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.522980    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl6jv\" (UniqueName: \"kubernetes.io/projected/9675c1ec-78db-4511-8621-ff9ab01f822a-kube-api-access-zl6jv\") pod \"payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27\" (UID: \"9675c1ec-78db-4511-8621-ff9ab01f822a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:30:34.561910 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.561890    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"]
Sep 16 18:30:34.565141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.565127    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:30:34.575302 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.575280    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"]
Sep 16 18:30:34.623998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.623968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9675c1ec-78db-4511-8621-ff9ab01f822a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27\" (UID: \"9675c1ec-78db-4511-8621-ff9ab01f822a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:30:34.623998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.623997    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl6jv\" (UniqueName: \"kubernetes.io/projected/9675c1ec-78db-4511-8621-ff9ab01f822a-kube-api-access-zl6jv\") pod \"payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27\" (UID: \"9675c1ec-78db-4511-8621-ff9ab01f822a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:30:34.624132 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.624055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06550cad-4e3a-415e-b698-ba10dd9b8902-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b\" (UID: \"06550cad-4e3a-415e-b698-ba10dd9b8902\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:30:34.624208 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.624191    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx225\" (UniqueName: \"kubernetes.io/projected/06550cad-4e3a-415e-b698-ba10dd9b8902-kube-api-access-cx225\") pod \"payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b\" (UID: \"06550cad-4e3a-415e-b698-ba10dd9b8902\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:30:34.624630 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.624614    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9675c1ec-78db-4511-8621-ff9ab01f822a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27\" (UID: \"9675c1ec-78db-4511-8621-ff9ab01f822a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:30:34.634220 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.634200    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl6jv\" (UniqueName: \"kubernetes.io/projected/9675c1ec-78db-4511-8621-ff9ab01f822a-kube-api-access-zl6jv\") pod \"payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27\" (UID: \"9675c1ec-78db-4511-8621-ff9ab01f822a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:30:34.724963 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.724907    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06550cad-4e3a-415e-b698-ba10dd9b8902-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b\" (UID: \"06550cad-4e3a-415e-b698-ba10dd9b8902\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:30:34.725047 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.724965    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx225\" (UniqueName: \"kubernetes.io/projected/06550cad-4e3a-415e-b698-ba10dd9b8902-kube-api-access-cx225\") pod \"payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b\" (UID: \"06550cad-4e3a-415e-b698-ba10dd9b8902\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:30:34.725611 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.725570    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06550cad-4e3a-415e-b698-ba10dd9b8902-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b\" (UID: \"06550cad-4e3a-415e-b698-ba10dd9b8902\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:30:34.732163 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.732140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx225\" (UniqueName: \"kubernetes.io/projected/06550cad-4e3a-415e-b698-ba10dd9b8902-kube-api-access-cx225\") pod \"payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b\" (UID: \"06550cad-4e3a-415e-b698-ba10dd9b8902\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:30:34.817312 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.817292    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:30:34.874891 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.874854    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:30:34.912934 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.912901    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:34.995219 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:34.995155    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:35.101149 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:35.101128    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"]
Sep 16 18:30:35.102563 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:30:35.102541    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9675c1ec_78db_4511_8621_ff9ab01f822a.slice/crio-5460eef079c0f72f9a385988cbaae7c6dafcfbe97bec3c3b83a72a0d08455a9c WatchSource:0}: Error finding container 5460eef079c0f72f9a385988cbaae7c6dafcfbe97bec3c3b83a72a0d08455a9c: Status 404 returned error can't find the container with id 5460eef079c0f72f9a385988cbaae7c6dafcfbe97bec3c3b83a72a0d08455a9c
Sep 16 18:30:35.153419 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:35.153198    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"]
Sep 16 18:30:35.156494 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:30:35.156471    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06550cad_4e3a_415e_b698_ba10dd9b8902.slice/crio-9ce3a477484bd844efd1280b957281497a8c64d3d4d1b3d9a3931c043d5462eb WatchSource:0}: Error finding container 9ce3a477484bd844efd1280b957281497a8c64d3d4d1b3d9a3931c043d5462eb: Status 404 returned error can't find the container with id 9ce3a477484bd844efd1280b957281497a8c64d3d4d1b3d9a3931c043d5462eb
Sep 16 18:30:35.953077 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:35.953025    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27" event={"ID":"9675c1ec-78db-4511-8621-ff9ab01f822a","Type":"ContainerStarted","Data":"511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87"}
Sep 16 18:30:35.953300 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:35.953093    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27" event={"ID":"9675c1ec-78db-4511-8621-ff9ab01f822a","Type":"ContainerStarted","Data":"5460eef079c0f72f9a385988cbaae7c6dafcfbe97bec3c3b83a72a0d08455a9c"}
Sep 16 18:30:35.953300 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:35.953224    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:30:35.954674 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:35.954654    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b" event={"ID":"06550cad-4e3a-415e-b698-ba10dd9b8902","Type":"ContainerStarted","Data":"48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1"}
Sep 16 18:30:35.954758 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:35.954687    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b" event={"ID":"06550cad-4e3a-415e-b698-ba10dd9b8902","Type":"ContainerStarted","Data":"9ce3a477484bd844efd1280b957281497a8c64d3d4d1b3d9a3931c043d5462eb"}
Sep 16 18:30:35.954812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:35.954802    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:30:35.975181 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:35.975137    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27" podStartSLOduration=1.975125062 podStartE2EDuration="1.975125062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:35.974361251 +0000 UTC m=+856.214150004" watchObservedRunningTime="2026-09-16 18:30:35.975125062 +0000 UTC m=+856.214913814"
Sep 16 18:30:35.997913 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:35.997875    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b" podStartSLOduration=1.99786341 podStartE2EDuration="1.99786341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:35.996851155 +0000 UTC m=+856.236639922" watchObservedRunningTime="2026-09-16 18:30:35.99786341 +0000 UTC m=+856.237652206"
Sep 16 18:30:36.020924 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.020900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:36.121858 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.121831    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:36.225735 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.225660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:36.342761 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.342723    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:36.430038 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.430008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:36.805754 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.805721    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:36.908843 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.908806    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:36.958335 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.958303    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"]
Sep 16 18:30:36.963363 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.963335    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:30:36.966291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.966267    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-6942a4-dockercfg-59p2n\""
Sep 16 18:30:36.966417 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.966298    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-6942a4\""
Sep 16 18:30:36.967968 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.967949    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"]
Sep 16 18:30:36.971377 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.971359    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:30:36.973578 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.973559    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"]
Sep 16 18:30:36.982124 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:36.982099    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"]
Sep 16 18:30:37.010718 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.010692    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:37.048096 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.048073    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h8vk\" (UniqueName: \"kubernetes.io/projected/9bf78544-63da-42b4-af9f-7dc6712ce018-kube-api-access-7h8vk\") pod \"payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm\" (UID: \"9bf78544-63da-42b4-af9f-7dc6712ce018\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:30:37.048182 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.048153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmrm4\" (UniqueName: \"kubernetes.io/projected/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-kube-api-access-nmrm4\") pod \"payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd\" (UID: \"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:30:37.048235 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.048224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd\" (UID: \"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:30:37.048280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.048264    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf78544-63da-42b4-af9f-7dc6712ce018-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm\" (UID: \"9bf78544-63da-42b4-af9f-7dc6712ce018\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:30:37.149093 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.149037    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd\" (UID: \"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:30:37.149093 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.149070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf78544-63da-42b4-af9f-7dc6712ce018-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm\" (UID: \"9bf78544-63da-42b4-af9f-7dc6712ce018\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:30:37.149421 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.149102    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h8vk\" (UniqueName: \"kubernetes.io/projected/9bf78544-63da-42b4-af9f-7dc6712ce018-kube-api-access-7h8vk\") pod \"payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm\" (UID: \"9bf78544-63da-42b4-af9f-7dc6712ce018\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:30:37.149421 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.149130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmrm4\" (UniqueName: \"kubernetes.io/projected/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-kube-api-access-nmrm4\") pod \"payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd\" (UID: \"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:30:37.149749 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.149733    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd\" (UID: \"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:30:37.149813 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.149796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf78544-63da-42b4-af9f-7dc6712ce018-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm\" (UID: \"9bf78544-63da-42b4-af9f-7dc6712ce018\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:30:37.160644 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.160627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmrm4\" (UniqueName: \"kubernetes.io/projected/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-kube-api-access-nmrm4\") pod \"payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd\" (UID: \"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:30:37.161740 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.161718    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h8vk\" (UniqueName: \"kubernetes.io/projected/9bf78544-63da-42b4-af9f-7dc6712ce018-kube-api-access-7h8vk\") pod \"payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm\" (UID: \"9bf78544-63da-42b4-af9f-7dc6712ce018\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:30:37.277604 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.277570    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:30:37.285657 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.285636    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:30:37.321611 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.321559    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:37.555778 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.555744    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"]
Sep 16 18:30:37.557973 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:30:37.557947    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bf78544_63da_42b4_af9f_7dc6712ce018.slice/crio-a019ce300e09307eb731163c3115bd4496907712f97cc777d2dbd7534f56eb29 WatchSource:0}: Error finding container a019ce300e09307eb731163c3115bd4496907712f97cc777d2dbd7534f56eb29: Status 404 returned error can't find the container with id a019ce300e09307eb731163c3115bd4496907712f97cc777d2dbd7534f56eb29
Sep 16 18:30:37.567256 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.567236    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"]
Sep 16 18:30:37.570756 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:30:37.570726    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d8f3f3_3f6b_4440_aaa5_0c16e8104c18.slice/crio-a93e75893aedec90d62b54702b227b2c02c5624f684aa70341d2c5616c18a375 WatchSource:0}: Error finding container a93e75893aedec90d62b54702b227b2c02c5624f684aa70341d2c5616c18a375: Status 404 returned error can't find the container with id a93e75893aedec90d62b54702b227b2c02c5624f684aa70341d2c5616c18a375
Sep 16 18:30:37.974385 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.974341    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd" event={"ID":"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18","Type":"ContainerStarted","Data":"5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3"}
Sep 16 18:30:37.974570 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.974404    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd" event={"ID":"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18","Type":"ContainerStarted","Data":"a93e75893aedec90d62b54702b227b2c02c5624f684aa70341d2c5616c18a375"}
Sep 16 18:30:37.974570 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.974511    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:30:37.976040 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.976014    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm" event={"ID":"9bf78544-63da-42b4-af9f-7dc6712ce018","Type":"ContainerStarted","Data":"b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840"}
Sep 16 18:30:37.976151 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.976052    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm" event={"ID":"9bf78544-63da-42b4-af9f-7dc6712ce018","Type":"ContainerStarted","Data":"a019ce300e09307eb731163c3115bd4496907712f97cc777d2dbd7534f56eb29"}
Sep 16 18:30:37.976151 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.976077    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:30:37.992768 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:37.992728    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd" podStartSLOduration=1.992715827 podStartE2EDuration="1.992715827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:37.991167839 +0000 UTC m=+858.230956590" watchObservedRunningTime="2026-09-16 18:30:37.992715827 +0000 UTC m=+858.232504610"
Sep 16 18:30:38.009637 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:38.009570    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm" podStartSLOduration=2.009557781 podStartE2EDuration="2.009557781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:38.008908288 +0000 UTC m=+858.248697041" watchObservedRunningTime="2026-09-16 18:30:38.009557781 +0000 UTC m=+858.249346532"
Sep 16 18:30:38.283322 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:38.283233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:38.618944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:38.618876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:38.717542 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:38.717513    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:38.851552 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:38.851525    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:39.189105 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:39.189074    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:40.190267 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.190229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:40.196769 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.196744    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"]
Sep 16 18:30:40.200331 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.200313    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:30:40.202840 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.202820    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-a24d09-dockercfg-rr8mx\""
Sep 16 18:30:40.202942 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.202884    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-a24d09\""
Sep 16 18:30:40.211291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.211270    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"]
Sep 16 18:30:40.279448 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.279417    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mwhn\" (UniqueName: \"kubernetes.io/projected/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-kube-api-access-5mwhn\") pod \"payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk\" (UID: \"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:30:40.279584 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.279495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk\" (UID: \"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:30:40.285510 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.285482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:40.380292 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.380265    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk\" (UID: \"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:30:40.380420 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.380345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mwhn\" (UniqueName: \"kubernetes.io/projected/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-kube-api-access-5mwhn\") pod \"payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk\" (UID: \"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:30:40.380989 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.380972    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk\" (UID: \"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:30:40.389161 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.389143    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mwhn\" (UniqueName: \"kubernetes.io/projected/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-kube-api-access-5mwhn\") pod \"payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk\" (UID: \"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:30:40.511126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.511069    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:30:40.704878 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.704841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:40.793049 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.793027    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"]
Sep 16 18:30:40.795023 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:30:40.795000    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5b272f8_ce53_48c4_ac86_55a90ee5e4b4.slice/crio-c3949eade79bd139602f3fdd7b76246f426c0a1dcaf1452b43e31f0771d8df04 WatchSource:0}: Error finding container c3949eade79bd139602f3fdd7b76246f426c0a1dcaf1452b43e31f0771d8df04: Status 404 returned error can't find the container with id c3949eade79bd139602f3fdd7b76246f426c0a1dcaf1452b43e31f0771d8df04
Sep 16 18:30:40.993354 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.993304    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk" event={"ID":"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4","Type":"ContainerStarted","Data":"31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9"}
Sep 16 18:30:40.993515 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.993368    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk" event={"ID":"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4","Type":"ContainerStarted","Data":"c3949eade79bd139602f3fdd7b76246f426c0a1dcaf1452b43e31f0771d8df04"}
Sep 16 18:30:40.993515 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:40.993484    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:30:41.044890 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:41.044809    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk" podStartSLOduration=1.044795618 podStartE2EDuration="1.044795618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:41.042796149 +0000 UTC m=+861.282584902" watchObservedRunningTime="2026-09-16 18:30:41.044795618 +0000 UTC m=+861.284584369"
Sep 16 18:30:41.939428 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:41.939391    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:42.198410 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:42.198336    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:42.289923 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:42.289895    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:42.471042 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:42.470932    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:42.552723 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:42.552676    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:43.346416 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:43.346374    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:44.378239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:44.378212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:45.486816 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:45.486777    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:46.452443 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:46.452407    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:46.551893 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:46.551860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:46.818572 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:46.818481    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:46.969044 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:46.969012    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:30:46.969183 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:46.969094    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:30:47.435040 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:47.435005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:47.535511 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:47.535483    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:47.641677 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:47.641647    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:47.747839 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:47.747769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:47.846964 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:47.846939    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:47.963057 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:47.963014    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:48.070522 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.070446    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:48.162441 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.162398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:48.262667 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.262637    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:48.295162 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.295139    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"]
Sep 16 18:30:48.298557 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.298542    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:30:48.300973 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.300951    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-305998\""
Sep 16 18:30:48.301095 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.300973    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-305998-dockercfg-8gvjq\""
Sep 16 18:30:48.309240 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.309221    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"]
Sep 16 18:30:48.349866 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.349807    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9msbr\" (UniqueName: \"kubernetes.io/projected/e2defd82-a0b2-432f-baa7-9de2bee02dd6-kube-api-access-9msbr\") pod \"payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm\" (UID: \"e2defd82-a0b2-432f-baa7-9de2bee02dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:30:48.349958 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.349929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2defd82-a0b2-432f-baa7-9de2bee02dd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm\" (UID: \"e2defd82-a0b2-432f-baa7-9de2bee02dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:30:48.366998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.366962    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"]
Sep 16 18:30:48.370302 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.370286    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:30:48.390108 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.390087    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"]
Sep 16 18:30:48.450732 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.450706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9msbr\" (UniqueName: \"kubernetes.io/projected/e2defd82-a0b2-432f-baa7-9de2bee02dd6-kube-api-access-9msbr\") pod \"payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm\" (UID: \"e2defd82-a0b2-432f-baa7-9de2bee02dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:30:48.450846 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.450747    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a638c38f-29d6-47ac-a100-a3fca6a6fa00-plugins-config-volume\") pod \"payload-processing-e2e-autores-305998-5b649db649-vb576\" (UID: \"a638c38f-29d6-47ac-a100-a3fca6a6fa00\") " pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:30:48.450846 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.450774    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm4wx\" (UniqueName: \"kubernetes.io/projected/a638c38f-29d6-47ac-a100-a3fca6a6fa00-kube-api-access-dm4wx\") pod \"payload-processing-e2e-autores-305998-5b649db649-vb576\" (UID: \"a638c38f-29d6-47ac-a100-a3fca6a6fa00\") " pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:30:48.450923 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.450854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2defd82-a0b2-432f-baa7-9de2bee02dd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm\" (UID: \"e2defd82-a0b2-432f-baa7-9de2bee02dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:30:48.451476 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.451458    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2defd82-a0b2-432f-baa7-9de2bee02dd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm\" (UID: \"e2defd82-a0b2-432f-baa7-9de2bee02dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:30:48.460404 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.460377    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9msbr\" (UniqueName: \"kubernetes.io/projected/e2defd82-a0b2-432f-baa7-9de2bee02dd6-kube-api-access-9msbr\") pod \"payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm\" (UID: \"e2defd82-a0b2-432f-baa7-9de2bee02dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:30:48.551862 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.551834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a638c38f-29d6-47ac-a100-a3fca6a6fa00-plugins-config-volume\") pod \"payload-processing-e2e-autores-305998-5b649db649-vb576\" (UID: \"a638c38f-29d6-47ac-a100-a3fca6a6fa00\") " pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:30:48.552027 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.552008    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm4wx\" (UniqueName: \"kubernetes.io/projected/a638c38f-29d6-47ac-a100-a3fca6a6fa00-kube-api-access-dm4wx\") pod \"payload-processing-e2e-autores-305998-5b649db649-vb576\" (UID: \"a638c38f-29d6-47ac-a100-a3fca6a6fa00\") " pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:30:48.552662 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.552641    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a638c38f-29d6-47ac-a100-a3fca6a6fa00-plugins-config-volume\") pod \"payload-processing-e2e-autores-305998-5b649db649-vb576\" (UID: \"a638c38f-29d6-47ac-a100-a3fca6a6fa00\") " pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:30:48.570016 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.569990    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm4wx\" (UniqueName: \"kubernetes.io/projected/a638c38f-29d6-47ac-a100-a3fca6a6fa00-kube-api-access-dm4wx\") pod \"payload-processing-e2e-autores-305998-5b649db649-vb576\" (UID: \"a638c38f-29d6-47ac-a100-a3fca6a6fa00\") " pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:30:48.606536 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.606462    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:48.609148 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.609133    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:30:48.679917 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.679890    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:30:48.708015 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.707988    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:48.880794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.880771    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"]
Sep 16 18:30:48.882239 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:30:48.882212    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2defd82_a0b2_432f_baa7_9de2bee02dd6.slice/crio-8086d711156ff885412a93a96aadba93d185ef373e68f2dc99694fe6311cf2c0 WatchSource:0}: Error finding container 8086d711156ff885412a93a96aadba93d185ef373e68f2dc99694fe6311cf2c0: Status 404 returned error can't find the container with id 8086d711156ff885412a93a96aadba93d185ef373e68f2dc99694fe6311cf2c0
Sep 16 18:30:48.955401 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.955382    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"]
Sep 16 18:30:48.957756 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:30:48.957730    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda638c38f_29d6_47ac_a100_a3fca6a6fa00.slice/crio-1e09d70e0f83e8ae8e5f56cf71b2e47f5dccb8d2034222b39590c296f048e5b0 WatchSource:0}: Error finding container 1e09d70e0f83e8ae8e5f56cf71b2e47f5dccb8d2034222b39590c296f048e5b0: Status 404 returned error can't find the container with id 1e09d70e0f83e8ae8e5f56cf71b2e47f5dccb8d2034222b39590c296f048e5b0
Sep 16 18:30:48.983104 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.983076    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:30:48.983184 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:48.983150    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:30:49.036550 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.036525    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576" event={"ID":"a638c38f-29d6-47ac-a100-a3fca6a6fa00","Type":"ContainerStarted","Data":"1e09d70e0f83e8ae8e5f56cf71b2e47f5dccb8d2034222b39590c296f048e5b0"}
Sep 16 18:30:49.038660 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.038632    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm" event={"ID":"e2defd82-a0b2-432f-baa7-9de2bee02dd6","Type":"ContainerStarted","Data":"c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab"}
Sep 16 18:30:49.038764 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.038687    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm" event={"ID":"e2defd82-a0b2-432f-baa7-9de2bee02dd6","Type":"ContainerStarted","Data":"8086d711156ff885412a93a96aadba93d185ef373e68f2dc99694fe6311cf2c0"}
Sep 16 18:30:49.038833 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.038785    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:30:49.062474 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.062419    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm" podStartSLOduration=1.062399461 podStartE2EDuration="1.062399461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:49.060932379 +0000 UTC m=+869.300721132" watchObservedRunningTime="2026-09-16 18:30:49.062399461 +0000 UTC m=+869.302188214"
Sep 16 18:30:49.081820 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.081791    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:49.538325 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.538289    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"]
Sep 16 18:30:49.541983 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.541960    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:30:49.544450 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.544421    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-78d09e99-dockercfg-57t96\""
Sep 16 18:30:49.544750 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.544734    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-78d09e99\""
Sep 16 18:30:49.553323 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.553300    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"]
Sep 16 18:30:49.630401 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.630372    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:49.663565 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.663541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4jvg\" (UniqueName: \"kubernetes.io/projected/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-kube-api-access-z4jvg\") pod \"payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4\" (UID: \"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb\") " pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:30:49.663721 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.663698    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-plugins-config-volume\") pod \"payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4\" (UID: \"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb\") " pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:30:49.730669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.730644    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:49.764405 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.764386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-plugins-config-volume\") pod \"payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4\" (UID: \"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb\") " pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:30:49.764477 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.764421    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4jvg\" (UniqueName: \"kubernetes.io/projected/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-kube-api-access-z4jvg\") pod \"payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4\" (UID: \"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb\") " pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:30:49.765014 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.764999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-plugins-config-volume\") pod \"payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4\" (UID: \"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb\") " pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:30:49.772210 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.772189    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4jvg\" (UniqueName: \"kubernetes.io/projected/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-kube-api-access-z4jvg\") pod \"payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4\" (UID: \"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb\") " pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:30:49.855072 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.855027    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:30:49.932485 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:49.932452    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:50.046223 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:50.046193    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576" event={"ID":"a638c38f-29d6-47ac-a100-a3fca6a6fa00","Type":"ContainerStarted","Data":"6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de"}
Sep 16 18:30:50.046445 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:50.046427    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:30:50.065313 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:50.065270    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576" podStartSLOduration=2.065254021 podStartE2EDuration="2.065254021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:50.063616818 +0000 UTC m=+870.303405570" watchObservedRunningTime="2026-09-16 18:30:50.065254021 +0000 UTC m=+870.305042776"
Sep 16 18:30:50.127725 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:50.127675    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"]
Sep 16 18:30:50.130020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:30:50.129995    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb38d7142_bf2a_46f9_9ac5_67d8d9ad6cfb.slice/crio-e11cdc9017e39e968593136841d1c10f50db9931f03958e229ae437c66cb0534 WatchSource:0}: Error finding container e11cdc9017e39e968593136841d1c10f50db9931f03958e229ae437c66cb0534: Status 404 returned error can't find the container with id e11cdc9017e39e968593136841d1c10f50db9931f03958e229ae437c66cb0534
Sep 16 18:30:50.572484 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:50.572452    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:51.053239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:51.053204    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4" event={"ID":"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb","Type":"ContainerStarted","Data":"32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee"}
Sep 16 18:30:51.053644 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:51.053406    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:30:51.053644 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:51.053441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4" event={"ID":"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb","Type":"ContainerStarted","Data":"e11cdc9017e39e968593136841d1c10f50db9931f03958e229ae437c66cb0534"}
Sep 16 18:30:51.071520 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:51.071454    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4" podStartSLOduration=2.07144341 podStartE2EDuration="2.07144341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:51.069982298 +0000 UTC m=+871.309771050" watchObservedRunningTime="2026-09-16 18:30:51.07144341 +0000 UTC m=+871.311232216"
Sep 16 18:30:51.258885 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:51.258856    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:51.448372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:51.448345    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:51.546139 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:51.546114    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:52.000294 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:52.000266    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:30:52.369136 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:52.369059    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:52.459476 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:52.459445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:52.559496 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:52.559467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:52.662304 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:52.662277    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:53.667398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:53.667370    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:54.566260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:54.566231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:55.530399 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:55.530370    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:56.427171 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:56.427127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:56.529203 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:56.529176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:56.630454 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:56.630384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:56.884150 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:56.884069    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:56.984080 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:56.984048    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:57.091861 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:57.091822    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:57.497953 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:57.497920    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:58.399852 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:58.399814    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:58.500741 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:58.500710    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:58.599848 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:58.599814    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:59.321889 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:59.321855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:59.812002 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:30:59.811971    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:00.047837 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:00.047804    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:31:00.172127 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:00.172094    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:00.280112 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:00.280082    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:00.376988 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:00.376958    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:00.478933 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:00.478875    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:01.055226 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:01.055201    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:31:01.106104 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:01.106078    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:01.209013 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:01.208981    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:01.305350 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:01.305279    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:01.404744 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:01.404712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:01.507888 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:01.507857    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:02.060755 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:02.060731    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:31:02.229016 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:02.228969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:03.312273 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:03.312240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:04.421822 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:04.421791    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:05.305415 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:05.305384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:06.105744 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:06.105706    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:06.206411 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:06.206382    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:06.806743 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:06.806706    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:07.587597 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:07.587567    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:07.921633 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:07.921605    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"]
Sep 16 18:31:07.926698 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:07.926679    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"
Sep 16 18:31:07.929245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:07.929227    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:31:07.929245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:07.929238    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:31:07.933177 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:07.933154    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"]
Sep 16 18:31:08.107700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:08.107672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:08.118198 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:08.118168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f44967df-5b7b-4cb1-a91b-58abac2738e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4\" (UID: \"f44967df-5b7b-4cb1-a91b-58abac2738e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"
Sep 16 18:31:08.176507 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:08.176447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:08.219468 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:08.219435    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f44967df-5b7b-4cb1-a91b-58abac2738e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4\" (UID: \"f44967df-5b7b-4cb1-a91b-58abac2738e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"
Sep 16 18:31:08.220005 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:08.219986    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f44967df-5b7b-4cb1-a91b-58abac2738e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4\" (UID: \"f44967df-5b7b-4cb1-a91b-58abac2738e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"
Sep 16 18:31:08.258606 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:08.258565    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"
Sep 16 18:31:08.408984 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:08.408949    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:08.529679 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:08.529660    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"]
Sep 16 18:31:08.531795 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:31:08.531768    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf44967df_5b7b_4cb1_a91b_58abac2738e1.slice/crio-b2e4e0a3566fc2362ca3e91529880a0b20d1ddcacfdc4ff27b7f0cb85d1f1864 WatchSource:0}: Error finding container b2e4e0a3566fc2362ca3e91529880a0b20d1ddcacfdc4ff27b7f0cb85d1f1864: Status 404 returned error can't find the container with id b2e4e0a3566fc2362ca3e91529880a0b20d1ddcacfdc4ff27b7f0cb85d1f1864
Sep 16 18:31:09.149211 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:09.149181    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4" event={"ID":"f44967df-5b7b-4cb1-a91b-58abac2738e1","Type":"ContainerStarted","Data":"b2e4e0a3566fc2362ca3e91529880a0b20d1ddcacfdc4ff27b7f0cb85d1f1864"}
Sep 16 18:31:09.172422 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:09.172393    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:10.010600 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:10.010554    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:10.788758 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:10.788727    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:11.604363 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:11.604337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:12.168289 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:12.168265    2562 generic.go:356] "Generic (PLEG): container finished" podID="f44967df-5b7b-4cb1-a91b-58abac2738e1" containerID="726f0004abf5eef5ad2c87dfb1f79982937dee41a80bb7a7964dcd0756deaf3d" exitCode=0
Sep 16 18:31:12.168581 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:12.168324    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4" event={"ID":"f44967df-5b7b-4cb1-a91b-58abac2738e1","Type":"ContainerDied","Data":"726f0004abf5eef5ad2c87dfb1f79982937dee41a80bb7a7964dcd0756deaf3d"}
Sep 16 18:31:12.601360 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:12.601328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:13.308871 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:13.308850    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"
Sep 16 18:31:13.342221 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:13.342188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:13.360630 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:13.360605    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f44967df-5b7b-4cb1-a91b-58abac2738e1-maas-api-service-ca\") pod \"f44967df-5b7b-4cb1-a91b-58abac2738e1\" (UID: \"f44967df-5b7b-4cb1-a91b-58abac2738e1\") "
Sep 16 18:31:13.360906 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:13.360874    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f44967df-5b7b-4cb1-a91b-58abac2738e1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f44967df-5b7b-4cb1-a91b-58abac2738e1" (UID: "f44967df-5b7b-4cb1-a91b-58abac2738e1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:31:13.461492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:13.461413    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f44967df-5b7b-4cb1-a91b-58abac2738e1-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:31:14.180299 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:14.180265    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4" event={"ID":"f44967df-5b7b-4cb1-a91b-58abac2738e1","Type":"ContainerDied","Data":"b2e4e0a3566fc2362ca3e91529880a0b20d1ddcacfdc4ff27b7f0cb85d1f1864"}
Sep 16 18:31:14.180299 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:14.180294    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"
Sep 16 18:31:14.180468 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:14.180349    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2e4e0a3566fc2362ca3e91529880a0b20d1ddcacfdc4ff27b7f0cb85d1f1864"
Sep 16 18:31:17.418623 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:17.418574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:23.785874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:23.785836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:24.364220 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.364183    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:24.395426 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.395396    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"]
Sep 16 18:31:24.395707 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.395674    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4" podUID="b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb" containerName="payload-processing" containerID="cri-o://32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee" gracePeriod=30
Sep 16 18:31:24.462215 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.462154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:24.563497 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.563446    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"]
Sep 16 18:31:24.564536 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.564447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:24.567039 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.567009    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-78d09e99-ng8m4"]
Sep 16 18:31:24.648502 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.648481    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:31:24.663300 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.663265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:24.730637 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.730583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:24.743237 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.743207    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4jvg\" (UniqueName: \"kubernetes.io/projected/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-kube-api-access-z4jvg\") pod \"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb\" (UID: \"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb\") "
Sep 16 18:31:24.743377 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.743305    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-plugins-config-volume\") pod \"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb\" (UID: \"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb\") "
Sep 16 18:31:24.743625 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.743602    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb" (UID: "b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:31:24.745402 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.745384    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-kube-api-access-z4jvg" (OuterVolumeSpecName: "kube-api-access-z4jvg") pod "b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb" (UID: "b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb"). InnerVolumeSpecName "kube-api-access-z4jvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:31:24.844683 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.844660    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:31:24.844683 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:24.844680    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4jvg\" (UniqueName: \"kubernetes.io/projected/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb-kube-api-access-z4jvg\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:31:25.096692 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.096669    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:25.229122 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.229085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:25.238020 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.237973    2562 generic.go:356] "Generic (PLEG): container finished" podID="b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb" containerID="32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee" exitCode=0
Sep 16 18:31:25.238128 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.238081    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"
Sep 16 18:31:25.238128 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.238109    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4" event={"ID":"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb","Type":"ContainerDied","Data":"32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee"}
Sep 16 18:31:25.238288 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.238174    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4" event={"ID":"b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb","Type":"ContainerDied","Data":"e11cdc9017e39e968593136841d1c10f50db9931f03958e229ae437c66cb0534"}
Sep 16 18:31:25.238288 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.238224    2562 scope.go:117] "RemoveContainer" containerID="32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee"
Sep 16 18:31:25.248086 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.248070    2562 scope.go:117] "RemoveContainer" containerID="32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee"
Sep 16 18:31:25.248385 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:31:25.248367    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee\": container with ID starting with 32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee not found: ID does not exist" containerID="32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee"
Sep 16 18:31:25.248452 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.248393    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee"} err="failed to get container status \"32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee\": rpc error: code = NotFound desc = could not find container \"32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee\": container with ID starting with 32196d1f367eb067bc1506f69039b97ba12638298028aefeb5438171c03ea4ee not found: ID does not exist"
Sep 16 18:31:25.262269 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.262245    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"]
Sep 16 18:31:25.264312 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.264290    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-78d09e99-944b84fd9-xfwn4"]
Sep 16 18:31:25.313890 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.313865    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:25.441185 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.441164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:25.777142 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.777061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:25.990509 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:25.990481    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:26.091263 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:26.091183    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:26.285440 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:26.285416    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb" path="/var/lib/kubelet/pods/b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb/volumes"
Sep 16 18:31:26.285748 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:26.285736    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f44967df-5b7b-4cb1-a91b-58abac2738e1" path="/var/lib/kubelet/pods/f44967df-5b7b-4cb1-a91b-58abac2738e1/volumes"
Sep 16 18:31:26.712682 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:26.712651    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:27.402863 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:27.402830    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:28.195020 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:28.194988    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:28.657877 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:28.657788    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:28.758244 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:28.758217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:29.121716 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:29.121693    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:30.101887 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.101847    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:30.157156 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.157126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:30.363478 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.363410    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:30.604110 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.604079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:30.678253 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.678227    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:30.713660 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.713638    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"]
Sep 16 18:31:30.714106 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.714089    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb" containerName="payload-processing"
Sep 16 18:31:30.714106 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.714107    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb" containerName="payload-processing"
Sep 16 18:31:30.714268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.714116    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f44967df-5b7b-4cb1-a91b-58abac2738e1" containerName="revoke-keys"
Sep 16 18:31:30.714268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.714122    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f44967df-5b7b-4cb1-a91b-58abac2738e1" containerName="revoke-keys"
Sep 16 18:31:30.714268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.714213    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b38d7142-bf2a-46f9-9ac5-67d8d9ad6cfb" containerName="payload-processing"
Sep 16 18:31:30.714268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.714223    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f44967df-5b7b-4cb1-a91b-58abac2738e1" containerName="revoke-keys"
Sep 16 18:31:30.718426 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.718411    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:31:30.720827 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.720799    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-78aa4077\""
Sep 16 18:31:30.720931 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.720845    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-78aa4077-dockercfg-c89fm\""
Sep 16 18:31:30.727482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.727462    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"]
Sep 16 18:31:30.794522 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.794487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb\" (UID: \"1b805a38-0fec-4224-ba83-8b2a72bc7b4f\") " pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:31:30.794649 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.794575    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnh57\" (UniqueName: \"kubernetes.io/projected/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-kube-api-access-jnh57\") pod \"payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb\" (UID: \"1b805a38-0fec-4224-ba83-8b2a72bc7b4f\") " pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:31:30.895127 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.895103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb\" (UID: \"1b805a38-0fec-4224-ba83-8b2a72bc7b4f\") " pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:31:30.895263 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.895154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnh57\" (UniqueName: \"kubernetes.io/projected/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-kube-api-access-jnh57\") pod \"payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb\" (UID: \"1b805a38-0fec-4224-ba83-8b2a72bc7b4f\") " pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:31:30.895790 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.895772    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb\" (UID: \"1b805a38-0fec-4224-ba83-8b2a72bc7b4f\") " pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:31:30.903371 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.903351    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnh57\" (UniqueName: \"kubernetes.io/projected/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-kube-api-access-jnh57\") pod \"payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb\" (UID: \"1b805a38-0fec-4224-ba83-8b2a72bc7b4f\") " pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:31:30.983561 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:30.983501    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:31.030457 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:31.030428    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:31:31.300640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:31.300617    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"]
Sep 16 18:31:31.302552 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:31:31.302529    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b805a38_0fec_4224_ba83_8b2a72bc7b4f.slice/crio-39e2f78e7c0caf25feed67d15ac1778bfab50faa200d5fb13a09a5334c15d837 WatchSource:0}: Error finding container 39e2f78e7c0caf25feed67d15ac1778bfab50faa200d5fb13a09a5334c15d837: Status 404 returned error can't find the container with id 39e2f78e7c0caf25feed67d15ac1778bfab50faa200d5fb13a09a5334c15d837
Sep 16 18:31:31.674308 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:31.674282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:32.275721 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:32.275688    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb" event={"ID":"1b805a38-0fec-4224-ba83-8b2a72bc7b4f","Type":"ContainerStarted","Data":"8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0"}
Sep 16 18:31:32.275886 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:32.275745    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb" event={"ID":"1b805a38-0fec-4224-ba83-8b2a72bc7b4f","Type":"ContainerStarted","Data":"39e2f78e7c0caf25feed67d15ac1778bfab50faa200d5fb13a09a5334c15d837"}
Sep 16 18:31:32.275886 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:32.275805    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:31:32.292947 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:32.292909    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb" podStartSLOduration=2.2928978669999998 podStartE2EDuration="2.292897867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:31:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:31:32.291207194 +0000 UTC m=+912.530995947" watchObservedRunningTime="2026-09-16 18:31:32.292897867 +0000 UTC m=+912.532686619"
Sep 16 18:31:32.624713 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:32.624653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:33.296708 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:33.296678    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:34.092623 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:34.092583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:43.281791 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:43.281764    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:31:50.217915 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:50.217876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:50.273477 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:50.273443    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:52.130192 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.130157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:52.389182 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.389107    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"]
Sep 16 18:31:52.392580 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.392561    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.395443 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.395423    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-6942a4\"/\"openshift-service-ca.crt\""
Sep 16 18:31:52.396391 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.396372    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-6942a4\"/\"kube-root-ca.crt\""
Sep 16 18:31:52.396520 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.396503    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-6942a4\"/\"simulator-w4-6942a4-kserve-self-signed-certs\""
Sep 16 18:31:52.396566 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.396558    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-6942a4\"/\"default-dockercfg-bmh8t\""
Sep 16 18:31:52.411300 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.411274    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"]
Sep 16 18:31:52.470004 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.469982    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-dshm\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.470107 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.470016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-home\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.470107 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.470034    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-tmp-dir\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.470107 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.470061    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-model-cache\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.470232 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.470155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de0b960c-ca89-472a-be3e-675ee166b0ed-tls-certs\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.470232 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.470186    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg926\" (UniqueName: \"kubernetes.io/projected/de0b960c-ca89-472a-be3e-675ee166b0ed-kube-api-access-lg926\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.524456 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.524428    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:52.570966 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.570942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg926\" (UniqueName: \"kubernetes.io/projected/de0b960c-ca89-472a-be3e-675ee166b0ed-kube-api-access-lg926\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.571075 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.571040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-dshm\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.571132 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.571077    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-home\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.571132 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.571097    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-tmp-dir\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.571132 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.571128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-model-cache\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.571283 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.571163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de0b960c-ca89-472a-be3e-675ee166b0ed-tls-certs\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.571463 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.571443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-home\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.571555 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.571535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-model-cache\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.571624 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.571563    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-tmp-dir\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.573364 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.573347    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-dshm\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.573659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.573642    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de0b960c-ca89-472a-be3e-675ee166b0ed-tls-certs\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.583093 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.583075    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg926\" (UniqueName: \"kubernetes.io/projected/de0b960c-ca89-472a-be3e-675ee166b0ed-kube-api-access-lg926\") pod \"simulator-w4-6942a4-kserve-b68b86b55-kb6b2\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") " pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.701942 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.701920    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:31:52.979124 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:52.978957    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"]
Sep 16 18:31:52.981786 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:31:52.981765    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde0b960c_ca89_472a_be3e_675ee166b0ed.slice/crio-1c3ae47f7a6066f2163b4e5a159b60515dd24c9aff639f40d6e71ce9351bd78a WatchSource:0}: Error finding container 1c3ae47f7a6066f2163b4e5a159b60515dd24c9aff639f40d6e71ce9351bd78a: Status 404 returned error can't find the container with id 1c3ae47f7a6066f2163b4e5a159b60515dd24c9aff639f40d6e71ce9351bd78a
Sep 16 18:31:53.000371 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.000342    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:53.378981 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.378913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2" event={"ID":"de0b960c-ca89-472a-be3e-675ee166b0ed","Type":"ContainerStarted","Data":"cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0"}
Sep 16 18:31:53.378981 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.378951    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2" event={"ID":"de0b960c-ca89-472a-be3e-675ee166b0ed","Type":"ContainerStarted","Data":"1c3ae47f7a6066f2163b4e5a159b60515dd24c9aff639f40d6e71ce9351bd78a"}
Sep 16 18:31:53.396787 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.396743    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2" podStartSLOduration=1.396729854 podStartE2EDuration="1.396729854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:31:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:31:53.396433136 +0000 UTC m=+933.636221886" watchObservedRunningTime="2026-09-16 18:31:53.396729854 +0000 UTC m=+933.636518606"
Sep 16 18:31:53.703046 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.703021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:53.719691 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.719664    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"]
Sep 16 18:31:53.723280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.723260    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"
Sep 16 18:31:53.725431 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.725412    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:31:53.725524 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.725467    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:31:53.730890 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.730868    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"]
Sep 16 18:31:53.882301 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.882276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/931b8e95-f516-4b6a-a423-d325d51bb003-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm\" (UID: \"931b8e95-f516-4b6a-a423-d325d51bb003\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"
Sep 16 18:31:53.983191 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.983135    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/931b8e95-f516-4b6a-a423-d325d51bb003-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm\" (UID: \"931b8e95-f516-4b6a-a423-d325d51bb003\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"
Sep 16 18:31:53.983673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:53.983656    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/931b8e95-f516-4b6a-a423-d325d51bb003-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm\" (UID: \"931b8e95-f516-4b6a-a423-d325d51bb003\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"
Sep 16 18:31:54.036007 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:54.035989    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"
Sep 16 18:31:54.310528 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:54.310508    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"]
Sep 16 18:31:54.312262 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:31:54.312237    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931b8e95_f516_4b6a_a423_d325d51bb003.slice/crio-8745d328d2750217990e3d922732f6bc10bbfda04f37721b4c71f36fdb7f8835 WatchSource:0}: Error finding container 8745d328d2750217990e3d922732f6bc10bbfda04f37721b4c71f36fdb7f8835: Status 404 returned error can't find the container with id 8745d328d2750217990e3d922732f6bc10bbfda04f37721b4c71f36fdb7f8835
Sep 16 18:31:54.390183 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:54.390148    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm" event={"ID":"931b8e95-f516-4b6a-a423-d325d51bb003","Type":"ContainerStarted","Data":"8745d328d2750217990e3d922732f6bc10bbfda04f37721b4c71f36fdb7f8835"}
Sep 16 18:31:54.597945 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:54.597881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:55.304820 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:55.304736    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:55.396345 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:55.396318    2562 generic.go:356] "Generic (PLEG): container finished" podID="931b8e95-f516-4b6a-a423-d325d51bb003" containerID="c0ad52bdaf365fd633a5f225ffdad5f1d5a3f15ca9084bccf9a558cfea4b38a3" exitCode=0
Sep 16 18:31:55.396649 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:55.396389    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm" event={"ID":"931b8e95-f516-4b6a-a423-d325d51bb003","Type":"ContainerDied","Data":"c0ad52bdaf365fd633a5f225ffdad5f1d5a3f15ca9084bccf9a558cfea4b38a3"}
Sep 16 18:31:56.030481 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:56.030453    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:56.538164 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:56.538111    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"
Sep 16 18:31:56.707420 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:56.707393    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/931b8e95-f516-4b6a-a423-d325d51bb003-maas-api-service-ca\") pod \"931b8e95-f516-4b6a-a423-d325d51bb003\" (UID: \"931b8e95-f516-4b6a-a423-d325d51bb003\") "
Sep 16 18:31:56.707732 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:56.707711    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/931b8e95-f516-4b6a-a423-d325d51bb003-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "931b8e95-f516-4b6a-a423-d325d51bb003" (UID: "931b8e95-f516-4b6a-a423-d325d51bb003"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:31:56.746819 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:56.746787    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:56.808033 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:56.807974    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/931b8e95-f516-4b6a-a423-d325d51bb003-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:31:57.408203 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:57.408179    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm" event={"ID":"931b8e95-f516-4b6a-a423-d325d51bb003","Type":"ContainerDied","Data":"8745d328d2750217990e3d922732f6bc10bbfda04f37721b4c71f36fdb7f8835"}
Sep 16 18:31:57.408203 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:57.408195    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"
Sep 16 18:31:57.408362 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:57.408221    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8745d328d2750217990e3d922732f6bc10bbfda04f37721b4c71f36fdb7f8835"
Sep 16 18:31:57.454237 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:57.454204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:58.170958 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:58.170930    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:58.887078 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:58.887047    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:59.686416 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.686357    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"]
Sep 16 18:31:59.686969 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.686777    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931b8e95-f516-4b6a-a423-d325d51bb003" containerName="revoke-keys"
Sep 16 18:31:59.686969 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.686789    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="931b8e95-f516-4b6a-a423-d325d51bb003" containerName="revoke-keys"
Sep 16 18:31:59.686969 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.686867    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="931b8e95-f516-4b6a-a423-d325d51bb003" containerName="revoke-keys"
Sep 16 18:31:59.689887 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.689866    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.692307 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.692281    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-305998\"/\"kube-root-ca.crt\""
Sep 16 18:31:59.692430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.692304    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-305998\"/\"openshift-service-ca.crt\""
Sep 16 18:31:59.692430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.692285    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-305998\"/\"default-dockercfg-g4jsb\""
Sep 16 18:31:59.693245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.693226    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-305998\"/\"autores-model-305998-kserve-self-signed-certs\""
Sep 16 18:31:59.701796 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.701774    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"]
Sep 16 18:31:59.721803 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.721773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:59.833252 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.833223    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqhlj\" (UniqueName: \"kubernetes.io/projected/1928dab2-eb8a-4de9-83be-b55941c67a43-kube-api-access-jqhlj\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.833398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.833263    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-tmp-dir\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.833398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.833322    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-dshm\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.833398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.833370    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1928dab2-eb8a-4de9-83be-b55941c67a43-tls-certs\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.833553 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.833401    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-home\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.833553 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.833418    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-model-cache\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.926172 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.926134    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:59.934065 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.934035    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqhlj\" (UniqueName: \"kubernetes.io/projected/1928dab2-eb8a-4de9-83be-b55941c67a43-kube-api-access-jqhlj\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.934200 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.934078    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-tmp-dir\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.934200 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.934126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-dshm\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.934200 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.934164    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1928dab2-eb8a-4de9-83be-b55941c67a43-tls-certs\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.934359 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.934206    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-home\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.934359 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.934232    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-model-cache\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.934631 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.934582    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-tmp-dir\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.934724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.934661    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-home\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.934724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.934704    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-model-cache\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.936519 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.936465    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-dshm\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.937001 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.936983    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1928dab2-eb8a-4de9-83be-b55941c67a43-tls-certs\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:31:59.945895 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:31:59.945876    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqhlj\" (UniqueName: \"kubernetes.io/projected/1928dab2-eb8a-4de9-83be-b55941c67a43-kube-api-access-jqhlj\") pod \"autores-model-305998-kserve-569d87bb67-9wscj\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") " pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:32:00.000541 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.000511    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:32:00.295086 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.291095    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"]
Sep 16 18:32:00.439611 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.436919    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj" event={"ID":"1928dab2-eb8a-4de9-83be-b55941c67a43","Type":"ContainerStarted","Data":"0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d"}
Sep 16 18:32:00.439611 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.436969    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj" event={"ID":"1928dab2-eb8a-4de9-83be-b55941c67a43","Type":"ContainerStarted","Data":"55c833bb64bb084992ad4997588d2107bb84a92cbd2b32bd3137b29e3a82266a"}
Sep 16 18:32:00.627604 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.627503    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:00.652011 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.651962    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj" podStartSLOduration=1.651944444 podStartE2EDuration="1.651944444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:31:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:00.464661824 +0000 UTC m=+940.704450578" watchObservedRunningTime="2026-09-16 18:32:00.651944444 +0000 UTC m=+940.891733195"
Sep 16 18:32:00.654281 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.654256    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"]
Sep 16 18:32:00.654503 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.654470    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb" podUID="1b805a38-0fec-4224-ba83-8b2a72bc7b4f" containerName="payload-processing" containerID="cri-o://8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0" gracePeriod=30
Sep 16 18:32:00.740860 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.740831    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:00.842823 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.842796    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"]
Sep 16 18:32:00.849121 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.849095    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78aa4077-thxtm"]
Sep 16 18:32:00.892022 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:00.892002    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:32:01.050136 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.050108    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-plugins-config-volume\") pod \"1b805a38-0fec-4224-ba83-8b2a72bc7b4f\" (UID: \"1b805a38-0fec-4224-ba83-8b2a72bc7b4f\") "
Sep 16 18:32:01.050275 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.050151    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnh57\" (UniqueName: \"kubernetes.io/projected/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-kube-api-access-jnh57\") pod \"1b805a38-0fec-4224-ba83-8b2a72bc7b4f\" (UID: \"1b805a38-0fec-4224-ba83-8b2a72bc7b4f\") "
Sep 16 18:32:01.050425 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.050405    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1b805a38-0fec-4224-ba83-8b2a72bc7b4f" (UID: "1b805a38-0fec-4224-ba83-8b2a72bc7b4f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:32:01.052224 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.052201    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-kube-api-access-jnh57" (OuterVolumeSpecName: "kube-api-access-jnh57") pod "1b805a38-0fec-4224-ba83-8b2a72bc7b4f" (UID: "1b805a38-0fec-4224-ba83-8b2a72bc7b4f"). InnerVolumeSpecName "kube-api-access-jnh57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:32:01.151412 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.151352    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:01.151412 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.151376    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnh57\" (UniqueName: \"kubernetes.io/projected/1b805a38-0fec-4224-ba83-8b2a72bc7b4f-kube-api-access-jnh57\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:01.443316 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.443293    2562 generic.go:356] "Generic (PLEG): container finished" podID="1b805a38-0fec-4224-ba83-8b2a72bc7b4f" containerID="8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0" exitCode=0
Sep 16 18:32:01.443459 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.443344    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"
Sep 16 18:32:01.443459 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.443383    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb" event={"ID":"1b805a38-0fec-4224-ba83-8b2a72bc7b4f","Type":"ContainerDied","Data":"8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0"}
Sep 16 18:32:01.443459 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.443423    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb" event={"ID":"1b805a38-0fec-4224-ba83-8b2a72bc7b4f","Type":"ContainerDied","Data":"39e2f78e7c0caf25feed67d15ac1778bfab50faa200d5fb13a09a5334c15d837"}
Sep 16 18:32:01.443459 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.443443    2562 scope.go:117] "RemoveContainer" containerID="8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0"
Sep 16 18:32:01.452897 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.452879    2562 scope.go:117] "RemoveContainer" containerID="8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0"
Sep 16 18:32:01.453151 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:32:01.453134    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0\": container with ID starting with 8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0 not found: ID does not exist" containerID="8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0"
Sep 16 18:32:01.453217 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.453158    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0"} err="failed to get container status \"8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0\": rpc error: code = NotFound desc = could not find container \"8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0\": container with ID starting with 8f0af4e5b080ce9b8ea520558991787e0e5ae9b3a269a4e6162949ecfba748c0 not found: ID does not exist"
Sep 16 18:32:01.479297 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.479277    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"]
Sep 16 18:32:01.496724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:01.496700    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-78aa4077-5df9f6b778-bg4mb"]
Sep 16 18:32:02.286394 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:02.286360    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b805a38-0fec-4224-ba83-8b2a72bc7b4f" path="/var/lib/kubelet/pods/1b805a38-0fec-4224-ba83-8b2a72bc7b4f/volumes"
Sep 16 18:32:02.286771 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:02.286686    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="931b8e95-f516-4b6a-a423-d325d51bb003" path="/var/lib/kubelet/pods/931b8e95-f516-4b6a-a423-d325d51bb003/volumes"
Sep 16 18:32:02.290873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:02.290848    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:02.702403 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:02.702384    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:32:02.702638 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:02.702621    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:32:02.714433 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:02.714414    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:32:03.072842 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:03.072766    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:03.464868 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:03.464843    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:32:03.762469 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:03.762393    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:04.702018 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:04.701984    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:04.931883 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:04.931850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:05.029407 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:05.029329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:05.566115 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:05.566084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:06.309342 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:06.309311    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:06.409540 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:06.409515    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:06.867561 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:06.867520    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:06.934997 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:06.934957    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:07.322373 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:07.322348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:08.001885 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:08.001852    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:08.751751 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:08.751722    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:09.589294 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:09.589260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:10.000945 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:10.000914    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:32:10.001302 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:10.000956    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:32:10.013067 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:10.013037    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:32:10.500630 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:10.500608    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:32:10.601732 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:10.601696    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:18.727702 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.727669    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"]
Sep 16 18:32:18.728067 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.728055    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b805a38-0fec-4224-ba83-8b2a72bc7b4f" containerName="payload-processing"
Sep 16 18:32:18.728067 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.728068    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b805a38-0fec-4224-ba83-8b2a72bc7b4f" containerName="payload-processing"
Sep 16 18:32:18.728149 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.728134    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b805a38-0fec-4224-ba83-8b2a72bc7b4f" containerName="payload-processing"
Sep 16 18:32:18.731210 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.731194    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.734057 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.734042    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a24d09\"/\"simulator-w0-a24d09-kserve-self-signed-certs\""
Sep 16 18:32:18.735120 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.735102    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-a24d09\"/\"kube-root-ca.crt\""
Sep 16 18:32:18.735181 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.735135    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-a24d09\"/\"openshift-service-ca.crt\""
Sep 16 18:32:18.735181 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.735164    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a24d09\"/\"default-dockercfg-m4w58\""
Sep 16 18:32:18.741987 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.741966    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"]
Sep 16 18:32:18.797898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.797877    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-model-cache\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.798001 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.797906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-dshm\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.798001 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.797929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-home\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.798001 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.797964    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-tmp-dir\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.798154 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.798007    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05112bb4-8288-4eb5-84ab-8e901a922f66-tls-certs\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.798154 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.798038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r82wf\" (UniqueName: \"kubernetes.io/projected/05112bb4-8288-4eb5-84ab-8e901a922f66-kube-api-access-r82wf\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.895284 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.895263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:18.899284 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.899263    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-model-cache\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.899374 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.899293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-dshm\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.899374 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.899317    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-home\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.899374 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.899341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-tmp-dir\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.899511 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.899445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05112bb4-8288-4eb5-84ab-8e901a922f66-tls-certs\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.899511 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.899497    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r82wf\" (UniqueName: \"kubernetes.io/projected/05112bb4-8288-4eb5-84ab-8e901a922f66-kube-api-access-r82wf\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.899628 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.899619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-model-cache\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.899703 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.899685    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-tmp-dir\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.899788 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.899770    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-home\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.901543 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.901528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-dshm\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.901853 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.901836    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05112bb4-8288-4eb5-84ab-8e901a922f66-tls-certs\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:18.907783 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:18.907764    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r82wf\" (UniqueName: \"kubernetes.io/projected/05112bb4-8288-4eb5-84ab-8e901a922f66-kube-api-access-r82wf\") pod \"simulator-w0-a24d09-kserve-766c8ffd5c-87tsn\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") " pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:19.039861 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:19.039808    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:19.312486 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:19.312463    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"]
Sep 16 18:32:19.314416 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:32:19.314391    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05112bb4_8288_4eb5_84ab_8e901a922f66.slice/crio-ab9fc634d902fdbc8a4072a1b3c813c0167ec08b9f18bc08c0c7471b70196ce7 WatchSource:0}: Error finding container ab9fc634d902fdbc8a4072a1b3c813c0167ec08b9f18bc08c0c7471b70196ce7: Status 404 returned error can't find the container with id ab9fc634d902fdbc8a4072a1b3c813c0167ec08b9f18bc08c0c7471b70196ce7
Sep 16 18:32:19.535947 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:19.535909    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn" event={"ID":"05112bb4-8288-4eb5-84ab-8e901a922f66","Type":"ContainerStarted","Data":"1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26"}
Sep 16 18:32:19.536088 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:19.535958    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn" event={"ID":"05112bb4-8288-4eb5-84ab-8e901a922f66","Type":"ContainerStarted","Data":"ab9fc634d902fdbc8a4072a1b3c813c0167ec08b9f18bc08c0c7471b70196ce7"}
Sep 16 18:32:19.559456 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:19.559413    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn" podStartSLOduration=1.559399924 podStartE2EDuration="1.559399924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:19.558040619 +0000 UTC m=+959.797829391" watchObservedRunningTime="2026-09-16 18:32:19.559399924 +0000 UTC m=+959.799188676"
Sep 16 18:32:25.033054 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:25.033001    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:28.498196 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:28.498150    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:29.039927 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:29.039895    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:29.040095 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:29.039940    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:29.052045 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:29.052023    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:29.600630 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:29.600574    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:32:29.631511 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:29.631477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:30.409398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:30.409367    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:30.664946 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:30.664919    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"]
Sep 16 18:32:30.669282 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:30.669266    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"
Sep 16 18:32:30.672136 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:30.672118    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:32:30.672302 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:30.672288    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:32:30.682887 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:30.682865    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"]
Sep 16 18:32:30.799903 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:30.799877    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz\" (UID: \"52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"
Sep 16 18:32:30.900941 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:30.900919    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz\" (UID: \"52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"
Sep 16 18:32:30.901417 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:30.901400    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz\" (UID: \"52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"
Sep 16 18:32:30.979184 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:30.979132    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"
Sep 16 18:32:31.249139 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:31.248994    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"]
Sep 16 18:32:31.251842 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:32:31.251813    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52932c9a_5c0b_4f69_bb9a_a81ad6ef7b25.slice/crio-f0a935b799be13227ca32da627c9f8d2232b574c147c5d7799d1352d313e72d6 WatchSource:0}: Error finding container f0a935b799be13227ca32da627c9f8d2232b574c147c5d7799d1352d313e72d6: Status 404 returned error can't find the container with id f0a935b799be13227ca32da627c9f8d2232b574c147c5d7799d1352d313e72d6
Sep 16 18:32:31.333452 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:31.333429    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:31.597866 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:31.597840    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz" event={"ID":"52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25","Type":"ContainerStarted","Data":"3f1bbf1f90513a22c2f9c7cb80b4d76db74f397338bbf77369fc95bc5c1d67af"}
Sep 16 18:32:31.597956 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:31.597888    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz" event={"ID":"52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25","Type":"ContainerStarted","Data":"f0a935b799be13227ca32da627c9f8d2232b574c147c5d7799d1352d313e72d6"}
Sep 16 18:32:31.617314 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:31.617272    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz" podStartSLOduration=1.61725457 podStartE2EDuration="1.61725457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:32:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:31.614772839 +0000 UTC m=+971.854561594" watchObservedRunningTime="2026-09-16 18:32:31.61725457 +0000 UTC m=+971.857043385"
Sep 16 18:32:32.355897 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:32.355826    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:32.604194 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:32.604170    2562 generic.go:356] "Generic (PLEG): container finished" podID="52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25" containerID="3f1bbf1f90513a22c2f9c7cb80b4d76db74f397338bbf77369fc95bc5c1d67af" exitCode=0
Sep 16 18:32:32.604322 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:32.604263    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz" event={"ID":"52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25","Type":"ContainerDied","Data":"3f1bbf1f90513a22c2f9c7cb80b4d76db74f397338bbf77369fc95bc5c1d67af"}
Sep 16 18:32:33.326123 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:33.326091    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:33.754106 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:33.754085    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"
Sep 16 18:32:33.823546 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:33.823526    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25-maas-api-service-ca\") pod \"52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25\" (UID: \"52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25\") "
Sep 16 18:32:33.823880 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:33.823856    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25" (UID: "52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:32:33.921038 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:33.921011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:33.925104 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:33.925087    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:34.615634 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.615555    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"
Sep 16 18:32:34.615634 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.615617    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz" event={"ID":"52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25","Type":"ContainerDied","Data":"f0a935b799be13227ca32da627c9f8d2232b574c147c5d7799d1352d313e72d6"}
Sep 16 18:32:34.615780 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.615653    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0a935b799be13227ca32da627c9f8d2232b574c147c5d7799d1352d313e72d6"
Sep 16 18:32:34.840795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.840755    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:34.971068 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.971040    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"]
Sep 16 18:32:34.971445 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.971433    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25" containerName="revoke-keys"
Sep 16 18:32:34.971494 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.971447    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25" containerName="revoke-keys"
Sep 16 18:32:34.971540 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.971511    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25" containerName="revoke-keys"
Sep 16 18:32:34.974738 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.974721    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"
Sep 16 18:32:34.977308 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.977289    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:32:34.977559 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.977544    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:32:34.985295 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:34.985277    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"]
Sep 16 18:32:35.035973 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:35.035946    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-305998-vchcw\" (UID: \"9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"
Sep 16 18:32:35.136831 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:35.136808    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-305998-vchcw\" (UID: \"9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"
Sep 16 18:32:35.137396 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:35.137371    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-305998-vchcw\" (UID: \"9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"
Sep 16 18:32:35.285151 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:35.285090    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"
Sep 16 18:32:35.560133 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:35.560112    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"]
Sep 16 18:32:35.562310 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:32:35.562280    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d5a8ca1_9b0d_47c4_a91e_37cee1b5bc8b.slice/crio-dbaac006c8afd343f0491019e081d760650a5cca7e2f10a88e16b535b01fd2dc WatchSource:0}: Error finding container dbaac006c8afd343f0491019e081d760650a5cca7e2f10a88e16b535b01fd2dc: Status 404 returned error can't find the container with id dbaac006c8afd343f0491019e081d760650a5cca7e2f10a88e16b535b01fd2dc
Sep 16 18:32:35.621841 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:35.621810    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw" event={"ID":"9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b","Type":"ContainerStarted","Data":"dbaac006c8afd343f0491019e081d760650a5cca7e2f10a88e16b535b01fd2dc"}
Sep 16 18:32:36.157235 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:36.157206    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:36.628101 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:36.628074    2562 generic.go:356] "Generic (PLEG): container finished" podID="9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b" containerID="396cd57bb495dba4ff3424bac23459d649fec724fddb076fc889033726b3722a" exitCode=0
Sep 16 18:32:36.628272 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:36.628160    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw" event={"ID":"9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b","Type":"ContainerDied","Data":"396cd57bb495dba4ff3424bac23459d649fec724fddb076fc889033726b3722a"}
Sep 16 18:32:37.020126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:37.020098    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:37.761353 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:37.761330    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"
Sep 16 18:32:37.859604 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:37.859572    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b-maas-api-service-ca\") pod \"9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b\" (UID: \"9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b\") "
Sep 16 18:32:37.859863 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:37.859845    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b" (UID: "9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:32:37.900752 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:37.900681    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:37.960734 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:37.960708    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:38.639861 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:38.639803    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"
Sep 16 18:32:38.639861 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:38.639837    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw" event={"ID":"9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b","Type":"ContainerDied","Data":"dbaac006c8afd343f0491019e081d760650a5cca7e2f10a88e16b535b01fd2dc"}
Sep 16 18:32:38.640031 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:38.639888    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbaac006c8afd343f0491019e081d760650a5cca7e2f10a88e16b535b01fd2dc"
Sep 16 18:32:38.889958 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:38.889890    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:39.603085 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:39.603052    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:40.407625 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:40.407571    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:40.612787 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:40.612752    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"]
Sep 16 18:32:40.636657 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:40.636621    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:41.219928 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:41.219890    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:41.358007 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:41.357980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:41.449430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:41.449398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:41.534396 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:41.534330    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:41.560541 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:41.560517    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"]
Sep 16 18:32:41.562867 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:41.562847    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69621cf4-zw9pz"]
Sep 16 18:32:41.592875 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:41.592854    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:41.973771 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:41.973742    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:42.286527 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:42.286470    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25" path="/var/lib/kubelet/pods/52932c9a-5c0b-4f69-bb9a-a81ad6ef7b25/volumes"
Sep 16 18:32:42.677021 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:42.676991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:42.776881 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:42.776856    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:43.222293 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:43.222254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:43.617528 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:43.617442    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:43.709850 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:43.709817    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:44.075375 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:44.075346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:44.804291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:44.804260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:45.676102 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:45.676073    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:46.091697 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:46.091612    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:46.192221 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:46.192188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:46.847931 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:46.847900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:47.221080 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:47.221046    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:48.236934 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:48.236893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:48.330944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:48.330916    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:48.816762 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:48.816735    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:48.981412 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:48.981377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:49.075278 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.075205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:49.244338 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.244314    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"]
Sep 16 18:32:49.244753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.244740    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b" containerName="revoke-keys"
Sep 16 18:32:49.244797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.244755    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b" containerName="revoke-keys"
Sep 16 18:32:49.244835 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.244821    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b" containerName="revoke-keys"
Sep 16 18:32:49.247886 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.247873    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:32:49.250999 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.250975    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-7e8b5de5-dockercfg-rzcq8\""
Sep 16 18:32:49.251155 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.251140    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-7e8b5de5\""
Sep 16 18:32:49.260212 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.260188    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"]
Sep 16 18:32:49.363711 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.363645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/877fef67-b5fa-429c-b894-34b1c1c4347a-plugins-config-volume\") pod \"payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5\" (UID: \"877fef67-b5fa-429c-b894-34b1c1c4347a\") " pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:32:49.363830 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.363715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qh5j\" (UniqueName: \"kubernetes.io/projected/877fef67-b5fa-429c-b894-34b1c1c4347a-kube-api-access-6qh5j\") pod \"payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5\" (UID: \"877fef67-b5fa-429c-b894-34b1c1c4347a\") " pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:32:49.465143 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.465118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qh5j\" (UniqueName: \"kubernetes.io/projected/877fef67-b5fa-429c-b894-34b1c1c4347a-kube-api-access-6qh5j\") pod \"payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5\" (UID: \"877fef67-b5fa-429c-b894-34b1c1c4347a\") " pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:32:49.465264 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.465192    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/877fef67-b5fa-429c-b894-34b1c1c4347a-plugins-config-volume\") pod \"payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5\" (UID: \"877fef67-b5fa-429c-b894-34b1c1c4347a\") " pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:32:49.465798 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.465779    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/877fef67-b5fa-429c-b894-34b1c1c4347a-plugins-config-volume\") pod \"payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5\" (UID: \"877fef67-b5fa-429c-b894-34b1c1c4347a\") " pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:32:49.485007 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.484978    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qh5j\" (UniqueName: \"kubernetes.io/projected/877fef67-b5fa-429c-b894-34b1c1c4347a-kube-api-access-6qh5j\") pod \"payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5\" (UID: \"877fef67-b5fa-429c-b894-34b1c1c4347a\") " pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:32:49.558432 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.558410    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:32:49.656122 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.656046    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:49.835722 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:49.835698    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"]
Sep 16 18:32:49.837116 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:32:49.837095    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877fef67_b5fa_429c_b894_34b1c1c4347a.slice/crio-ccf8b4fa54f50ab34b0bea9f774da59a6c54e7a41566c766a5d04a8e0fd5dc55 WatchSource:0}: Error finding container ccf8b4fa54f50ab34b0bea9f774da59a6c54e7a41566c766a5d04a8e0fd5dc55: Status 404 returned error can't find the container with id ccf8b4fa54f50ab34b0bea9f774da59a6c54e7a41566c766a5d04a8e0fd5dc55
Sep 16 18:32:50.560918 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:50.560881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:50.700487 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:50.700453    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5" event={"ID":"877fef67-b5fa-429c-b894-34b1c1c4347a","Type":"ContainerStarted","Data":"ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a"}
Sep 16 18:32:50.700664 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:50.700510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5" event={"ID":"877fef67-b5fa-429c-b894-34b1c1c4347a","Type":"ContainerStarted","Data":"ccf8b4fa54f50ab34b0bea9f774da59a6c54e7a41566c766a5d04a8e0fd5dc55"}
Sep 16 18:32:50.700664 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:50.700638    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:32:50.726844 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:50.726798    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5" podStartSLOduration=1.726782708 podStartE2EDuration="1.726782708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:32:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:50.725845135 +0000 UTC m=+990.965633899" watchObservedRunningTime="2026-09-16 18:32:50.726782708 +0000 UTC m=+990.966571460"
Sep 16 18:32:51.451330 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:51.451293    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:52.139181 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.139145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:52.180930 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.180901    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"]
Sep 16 18:32:52.181151 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.181122    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576" podUID="a638c38f-29d6-47ac-a100-a3fca6a6fa00" containerName="payload-processing" containerID="cri-o://6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de" gracePeriod=30
Sep 16 18:32:52.188867 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.188846    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"]
Sep 16 18:32:52.189028 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.189009    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm" podUID="e2defd82-a0b2-432f-baa7-9de2bee02dd6" containerName="payload-pre-processing" containerID="cri-o://c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab" gracePeriod=30
Sep 16 18:32:52.251451 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.251425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:52.346890 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.346854    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:52.447613 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.443022    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:52.482780 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.482761    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:32:52.485741 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.485724    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:32:52.593021 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.592997    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a638c38f-29d6-47ac-a100-a3fca6a6fa00-plugins-config-volume\") pod \"a638c38f-29d6-47ac-a100-a3fca6a6fa00\" (UID: \"a638c38f-29d6-47ac-a100-a3fca6a6fa00\") "
Sep 16 18:32:52.593147 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.593035    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm4wx\" (UniqueName: \"kubernetes.io/projected/a638c38f-29d6-47ac-a100-a3fca6a6fa00-kube-api-access-dm4wx\") pod \"a638c38f-29d6-47ac-a100-a3fca6a6fa00\" (UID: \"a638c38f-29d6-47ac-a100-a3fca6a6fa00\") "
Sep 16 18:32:52.593147 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.593110    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2defd82-a0b2-432f-baa7-9de2bee02dd6-plugins-config-volume\") pod \"e2defd82-a0b2-432f-baa7-9de2bee02dd6\" (UID: \"e2defd82-a0b2-432f-baa7-9de2bee02dd6\") "
Sep 16 18:32:52.593245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.593179    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9msbr\" (UniqueName: \"kubernetes.io/projected/e2defd82-a0b2-432f-baa7-9de2bee02dd6-kube-api-access-9msbr\") pod \"e2defd82-a0b2-432f-baa7-9de2bee02dd6\" (UID: \"e2defd82-a0b2-432f-baa7-9de2bee02dd6\") "
Sep 16 18:32:52.593320 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.593296    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a638c38f-29d6-47ac-a100-a3fca6a6fa00-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a638c38f-29d6-47ac-a100-a3fca6a6fa00" (UID: "a638c38f-29d6-47ac-a100-a3fca6a6fa00"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:32:52.593437 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.593423    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a638c38f-29d6-47ac-a100-a3fca6a6fa00-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:52.593539 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.593518    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2defd82-a0b2-432f-baa7-9de2bee02dd6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2defd82-a0b2-432f-baa7-9de2bee02dd6" (UID: "e2defd82-a0b2-432f-baa7-9de2bee02dd6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:32:52.595344 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.595322    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a638c38f-29d6-47ac-a100-a3fca6a6fa00-kube-api-access-dm4wx" (OuterVolumeSpecName: "kube-api-access-dm4wx") pod "a638c38f-29d6-47ac-a100-a3fca6a6fa00" (UID: "a638c38f-29d6-47ac-a100-a3fca6a6fa00"). InnerVolumeSpecName "kube-api-access-dm4wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:32:52.595439 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.595351    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2defd82-a0b2-432f-baa7-9de2bee02dd6-kube-api-access-9msbr" (OuterVolumeSpecName: "kube-api-access-9msbr") pod "e2defd82-a0b2-432f-baa7-9de2bee02dd6" (UID: "e2defd82-a0b2-432f-baa7-9de2bee02dd6"). InnerVolumeSpecName "kube-api-access-9msbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:32:52.622076 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.622045    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"]
Sep 16 18:32:52.632812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.632791    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-305998-vchcw"]
Sep 16 18:32:52.647633 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.647569    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:52.694409 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.694390    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2defd82-a0b2-432f-baa7-9de2bee02dd6-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:52.694409 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.694409    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9msbr\" (UniqueName: \"kubernetes.io/projected/e2defd82-a0b2-432f-baa7-9de2bee02dd6-kube-api-access-9msbr\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:52.694519 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.694422    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm4wx\" (UniqueName: \"kubernetes.io/projected/a638c38f-29d6-47ac-a100-a3fca6a6fa00-kube-api-access-dm4wx\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:52.713383 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.713358    2562 generic.go:356] "Generic (PLEG): container finished" podID="a638c38f-29d6-47ac-a100-a3fca6a6fa00" containerID="6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de" exitCode=0
Sep 16 18:32:52.713467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.713455    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"
Sep 16 18:32:52.713525 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.713506    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576" event={"ID":"a638c38f-29d6-47ac-a100-a3fca6a6fa00","Type":"ContainerDied","Data":"6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de"}
Sep 16 18:32:52.713581 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.713552    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576" event={"ID":"a638c38f-29d6-47ac-a100-a3fca6a6fa00","Type":"ContainerDied","Data":"1e09d70e0f83e8ae8e5f56cf71b2e47f5dccb8d2034222b39590c296f048e5b0"}
Sep 16 18:32:52.713656 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.713581    2562 scope.go:117] "RemoveContainer" containerID="6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de"
Sep 16 18:32:52.714951 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.714928    2562 generic.go:356] "Generic (PLEG): container finished" podID="e2defd82-a0b2-432f-baa7-9de2bee02dd6" containerID="c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab" exitCode=0
Sep 16 18:32:52.715030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.714984    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"
Sep 16 18:32:52.715030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.714986    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm" event={"ID":"e2defd82-a0b2-432f-baa7-9de2bee02dd6","Type":"ContainerDied","Data":"c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab"}
Sep 16 18:32:52.715122 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.715033    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm" event={"ID":"e2defd82-a0b2-432f-baa7-9de2bee02dd6","Type":"ContainerDied","Data":"8086d711156ff885412a93a96aadba93d185ef373e68f2dc99694fe6311cf2c0"}
Sep 16 18:32:52.723543 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.723531    2562 scope.go:117] "RemoveContainer" containerID="6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de"
Sep 16 18:32:52.723879 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:32:52.723857    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de\": container with ID starting with 6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de not found: ID does not exist" containerID="6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de"
Sep 16 18:32:52.723963 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.723884    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de"} err="failed to get container status \"6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de\": rpc error: code = NotFound desc = could not find container \"6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de\": container with ID starting with 6e4c3de9f73ce11d6178ff5609c790fd16c9e4f7573ab03852ff40c0196269de not found: ID does not exist"
Sep 16 18:32:52.723963 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.723899    2562 scope.go:117] "RemoveContainer" containerID="c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab"
Sep 16 18:32:52.732126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.732109    2562 scope.go:117] "RemoveContainer" containerID="c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab"
Sep 16 18:32:52.732450 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:32:52.732428    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab\": container with ID starting with c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab not found: ID does not exist" containerID="c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab"
Sep 16 18:32:52.732489 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.732461    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab"} err="failed to get container status \"c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab\": rpc error: code = NotFound desc = could not find container \"c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab\": container with ID starting with c62330c682f7d33382cbcc977d94f1e4f83b83d4a9293efd101c65635b1e4eab not found: ID does not exist"
Sep 16 18:32:52.757750 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.757723    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"]
Sep 16 18:32:52.779202 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.779157    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-305998-5b649db649-vb576"]
Sep 16 18:32:52.779931 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.779905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:52.801744 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.801722    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"]
Sep 16 18:32:52.810893 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:52.810876    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-305998-789f8bb46-xpxpm"]
Sep 16 18:32:53.058267 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:53.058236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:54.070467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:54.070434    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:54.285605 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:54.285572    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b" path="/var/lib/kubelet/pods/9d5a8ca1-9b0d-47c4-a91e-37cee1b5bc8b/volumes"
Sep 16 18:32:54.285957 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:54.285943    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a638c38f-29d6-47ac-a100-a3fca6a6fa00" path="/var/lib/kubelet/pods/a638c38f-29d6-47ac-a100-a3fca6a6fa00/volumes"
Sep 16 18:32:54.286252 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:54.286241    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2defd82-a0b2-432f-baa7-9de2bee02dd6" path="/var/lib/kubelet/pods/e2defd82-a0b2-432f-baa7-9de2bee02dd6/volumes"
Sep 16 18:32:55.255196 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:55.255164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:55.631308 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:55.631236    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj" podUID="1928dab2-eb8a-4de9-83be-b55941c67a43" containerName="main" containerID="cri-o://0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d" gracePeriod=45
Sep 16 18:32:55.877482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:55.877458    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:32:56.024541 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.024516    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-model-cache\") pod \"1928dab2-eb8a-4de9-83be-b55941c67a43\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") "
Sep 16 18:32:56.024719 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.024550    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-tmp-dir\") pod \"1928dab2-eb8a-4de9-83be-b55941c67a43\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") "
Sep 16 18:32:56.024719 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.024640    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-dshm\") pod \"1928dab2-eb8a-4de9-83be-b55941c67a43\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") "
Sep 16 18:32:56.024719 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.024703    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-home\") pod \"1928dab2-eb8a-4de9-83be-b55941c67a43\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") "
Sep 16 18:32:56.025064 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.024781    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1928dab2-eb8a-4de9-83be-b55941c67a43-tls-certs\") pod \"1928dab2-eb8a-4de9-83be-b55941c67a43\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") "
Sep 16 18:32:56.025064 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.024852    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqhlj\" (UniqueName: \"kubernetes.io/projected/1928dab2-eb8a-4de9-83be-b55941c67a43-kube-api-access-jqhlj\") pod \"1928dab2-eb8a-4de9-83be-b55941c67a43\" (UID: \"1928dab2-eb8a-4de9-83be-b55941c67a43\") "
Sep 16 18:32:56.025064 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.024769    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-model-cache" (OuterVolumeSpecName: "model-cache") pod "1928dab2-eb8a-4de9-83be-b55941c67a43" (UID: "1928dab2-eb8a-4de9-83be-b55941c67a43"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:56.025064 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.024855    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1928dab2-eb8a-4de9-83be-b55941c67a43" (UID: "1928dab2-eb8a-4de9-83be-b55941c67a43"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:56.025504 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.025071    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-home" (OuterVolumeSpecName: "home") pod "1928dab2-eb8a-4de9-83be-b55941c67a43" (UID: "1928dab2-eb8a-4de9-83be-b55941c67a43"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:56.025504 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.025241    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:56.025504 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.025259    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:56.025504 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.025274    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:56.027745 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.027721    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-dshm" (OuterVolumeSpecName: "dshm") pod "1928dab2-eb8a-4de9-83be-b55941c67a43" (UID: "1928dab2-eb8a-4de9-83be-b55941c67a43"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:32:56.027856 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.027807    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1928dab2-eb8a-4de9-83be-b55941c67a43-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1928dab2-eb8a-4de9-83be-b55941c67a43" (UID: "1928dab2-eb8a-4de9-83be-b55941c67a43"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:32:56.027940 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.027923    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1928dab2-eb8a-4de9-83be-b55941c67a43-kube-api-access-jqhlj" (OuterVolumeSpecName: "kube-api-access-jqhlj") pod "1928dab2-eb8a-4de9-83be-b55941c67a43" (UID: "1928dab2-eb8a-4de9-83be-b55941c67a43"). InnerVolumeSpecName "kube-api-access-jqhlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:32:56.126558 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.126512    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1928dab2-eb8a-4de9-83be-b55941c67a43-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:56.126558 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.126534    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1928dab2-eb8a-4de9-83be-b55941c67a43-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:56.126558 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.126544    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqhlj\" (UniqueName: \"kubernetes.io/projected/1928dab2-eb8a-4de9-83be-b55941c67a43-kube-api-access-jqhlj\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:32:56.128790 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.128759    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:56.739454 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.739424    2562 generic.go:356] "Generic (PLEG): container finished" podID="1928dab2-eb8a-4de9-83be-b55941c67a43" containerID="0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d" exitCode=0
Sep 16 18:32:56.739826 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.739457    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"
Sep 16 18:32:56.739826 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.739479    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj" event={"ID":"1928dab2-eb8a-4de9-83be-b55941c67a43","Type":"ContainerDied","Data":"0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d"}
Sep 16 18:32:56.739826 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.739515    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj" event={"ID":"1928dab2-eb8a-4de9-83be-b55941c67a43","Type":"ContainerDied","Data":"55c833bb64bb084992ad4997588d2107bb84a92cbd2b32bd3137b29e3a82266a"}
Sep 16 18:32:56.739826 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.739536    2562 scope.go:117] "RemoveContainer" containerID="0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d"
Sep 16 18:32:56.748643 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.748622    2562 scope.go:117] "RemoveContainer" containerID="0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d"
Sep 16 18:32:56.748915 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:32:56.748898    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d\": container with ID starting with 0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d not found: ID does not exist" containerID="0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d"
Sep 16 18:32:56.748987 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.748922    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d"} err="failed to get container status \"0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d\": rpc error: code = NotFound desc = could not find container \"0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d\": container with ID starting with 0826b634c825a4b2bf84bc06a8a225d9f4f8a770dc952d7c1a594e9bb908798d not found: ID does not exist"
Sep 16 18:32:56.756795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.756773    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"]
Sep 16 18:32:56.760380 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.760362    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-305998/autores-model-305998-kserve-569d87bb67-9wscj"]
Sep 16 18:32:56.896258 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:56.896229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:57.617844 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:57.617821    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:58.285366 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:58.285340    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1928dab2-eb8a-4de9-83be-b55941c67a43" path="/var/lib/kubelet/pods/1928dab2-eb8a-4de9-83be-b55941c67a43/volumes"
Sep 16 18:32:58.623973 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:58.623900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:59.495030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:59.494999    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:59.700986 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:59.700959    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:59.798529 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:32:59.798468    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:00.001183 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.001148    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:00.352968 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.352936    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:00.453249 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.453222    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:00.728815 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.728787    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"]
Sep 16 18:33:00.729195 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.729183    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2defd82-a0b2-432f-baa7-9de2bee02dd6" containerName="payload-pre-processing"
Sep 16 18:33:00.729240 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.729197    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2defd82-a0b2-432f-baa7-9de2bee02dd6" containerName="payload-pre-processing"
Sep 16 18:33:00.729240 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.729205    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1928dab2-eb8a-4de9-83be-b55941c67a43" containerName="main"
Sep 16 18:33:00.729240 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.729210    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1928dab2-eb8a-4de9-83be-b55941c67a43" containerName="main"
Sep 16 18:33:00.729240 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.729225    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a638c38f-29d6-47ac-a100-a3fca6a6fa00" containerName="payload-processing"
Sep 16 18:33:00.729240 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.729233    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a638c38f-29d6-47ac-a100-a3fca6a6fa00" containerName="payload-processing"
Sep 16 18:33:00.729402 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.729299    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2defd82-a0b2-432f-baa7-9de2bee02dd6" containerName="payload-pre-processing"
Sep 16 18:33:00.729402 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.729317    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a638c38f-29d6-47ac-a100-a3fca6a6fa00" containerName="payload-processing"
Sep 16 18:33:00.729402 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.729324    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1928dab2-eb8a-4de9-83be-b55941c67a43" containerName="main"
Sep 16 18:33:00.733832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.733816    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"
Sep 16 18:33:00.737744 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.737724    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:33:00.738042 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.738025    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:33:00.743975 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.743950    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"]
Sep 16 18:33:00.747056 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.747033    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:00.764357 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.764336    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e085821-c192-4c2f-827a-8f8ff2a64dd3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm\" (UID: \"4e085821-c192-4c2f-827a-8f8ff2a64dd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"
Sep 16 18:33:00.865346 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.865316    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e085821-c192-4c2f-827a-8f8ff2a64dd3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm\" (UID: \"4e085821-c192-4c2f-827a-8f8ff2a64dd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"
Sep 16 18:33:00.865905 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:00.865887    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e085821-c192-4c2f-827a-8f8ff2a64dd3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm\" (UID: \"4e085821-c192-4c2f-827a-8f8ff2a64dd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"
Sep 16 18:33:01.046638 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:01.046569    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"
Sep 16 18:33:01.290817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:01.290784    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:01.348787 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:01.348767    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"]
Sep 16 18:33:01.350914 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:33:01.350887    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e085821_c192_4c2f_827a_8f8ff2a64dd3.slice/crio-eb9f6f51049cabaa5c4b77ccc447f3b8a13442d73139b048c148e604e9b35099 WatchSource:0}: Error finding container eb9f6f51049cabaa5c4b77ccc447f3b8a13442d73139b048c148e604e9b35099: Status 404 returned error can't find the container with id eb9f6f51049cabaa5c4b77ccc447f3b8a13442d73139b048c148e604e9b35099
Sep 16 18:33:01.394727 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:01.394700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:01.708180 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:01.708161    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:33:01.770564 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:01.770524    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm" event={"ID":"4e085821-c192-4c2f-827a-8f8ff2a64dd3","Type":"ContainerStarted","Data":"5f84eaff4611a940f66347ae604fac7a06a59bbf074f3572f85535e9a5197aa1"}
Sep 16 18:33:01.770564 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:01.770569    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm" event={"ID":"4e085821-c192-4c2f-827a-8f8ff2a64dd3","Type":"ContainerStarted","Data":"eb9f6f51049cabaa5c4b77ccc447f3b8a13442d73139b048c148e604e9b35099"}
Sep 16 18:33:01.794283 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:01.794239    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm" podStartSLOduration=1.79422347 podStartE2EDuration="1.79422347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:33:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:33:01.792225989 +0000 UTC m=+1002.032014742" watchObservedRunningTime="2026-09-16 18:33:01.79422347 +0000 UTC m=+1002.034012222"
Sep 16 18:33:02.393156 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:02.393127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:02.776736 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:02.776707    2562 generic.go:356] "Generic (PLEG): container finished" podID="4e085821-c192-4c2f-827a-8f8ff2a64dd3" containerID="5f84eaff4611a940f66347ae604fac7a06a59bbf074f3572f85535e9a5197aa1" exitCode=0
Sep 16 18:33:02.777093 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:02.776746    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm" event={"ID":"4e085821-c192-4c2f-827a-8f8ff2a64dd3","Type":"ContainerDied","Data":"5f84eaff4611a940f66347ae604fac7a06a59bbf074f3572f85535e9a5197aa1"}
Sep 16 18:33:03.296841 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:03.296810    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:03.715533 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:03.715494    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:03.808759 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:03.808684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:03.913098 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:03.913077    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"
Sep 16 18:33:03.996951 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:03.996891    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e085821-c192-4c2f-827a-8f8ff2a64dd3-maas-api-service-ca\") pod \"4e085821-c192-4c2f-827a-8f8ff2a64dd3\" (UID: \"4e085821-c192-4c2f-827a-8f8ff2a64dd3\") "
Sep 16 18:33:03.997201 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:03.997181    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e085821-c192-4c2f-827a-8f8ff2a64dd3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e085821-c192-4c2f-827a-8f8ff2a64dd3" (UID: "4e085821-c192-4c2f-827a-8f8ff2a64dd3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:33:04.097871 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:04.097846    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e085821-c192-4c2f-827a-8f8ff2a64dd3-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:33:04.126092 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:04.126064    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:04.788167 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:04.788144    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"
Sep 16 18:33:04.788167 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:04.788158    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm" event={"ID":"4e085821-c192-4c2f-827a-8f8ff2a64dd3","Type":"ContainerDied","Data":"eb9f6f51049cabaa5c4b77ccc447f3b8a13442d73139b048c148e604e9b35099"}
Sep 16 18:33:04.788344 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:04.788200    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb9f6f51049cabaa5c4b77ccc447f3b8a13442d73139b048c148e604e9b35099"
Sep 16 18:33:05.110410 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:05.110328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:05.990921 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:05.990889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:06.886381 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:06.886346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:06.986183 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:06.986152    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:07.392341 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:07.392308    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:07.490612 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:07.490566    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:07.584985 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:07.584953    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:07.918033 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:07.918008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:08.918826 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:08.918799    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:10.092943 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:10.092915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:11.058161 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.058126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:11.122805 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.122777    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"]
Sep 16 18:33:11.123179 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.122988    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5" podUID="877fef67-b5fa-429c-b894-34b1c1c4347a" containerName="payload-processing" containerID="cri-o://ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a" gracePeriod=30
Sep 16 18:33:11.175277 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.175250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:11.295422 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.295397    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:11.422246 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.422221    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:33:11.437546 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.437515    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:11.460481 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.460455    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qh5j\" (UniqueName: \"kubernetes.io/projected/877fef67-b5fa-429c-b894-34b1c1c4347a-kube-api-access-6qh5j\") pod \"877fef67-b5fa-429c-b894-34b1c1c4347a\" (UID: \"877fef67-b5fa-429c-b894-34b1c1c4347a\") "
Sep 16 18:33:11.460634 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.460525    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/877fef67-b5fa-429c-b894-34b1c1c4347a-plugins-config-volume\") pod \"877fef67-b5fa-429c-b894-34b1c1c4347a\" (UID: \"877fef67-b5fa-429c-b894-34b1c1c4347a\") "
Sep 16 18:33:11.460846 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.460828    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/877fef67-b5fa-429c-b894-34b1c1c4347a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "877fef67-b5fa-429c-b894-34b1c1c4347a" (UID: "877fef67-b5fa-429c-b894-34b1c1c4347a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:33:11.462622 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.462601    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/877fef67-b5fa-429c-b894-34b1c1c4347a-kube-api-access-6qh5j" (OuterVolumeSpecName: "kube-api-access-6qh5j") pod "877fef67-b5fa-429c-b894-34b1c1c4347a" (UID: "877fef67-b5fa-429c-b894-34b1c1c4347a"). InnerVolumeSpecName "kube-api-access-6qh5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:33:11.476447 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.476426    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"]
Sep 16 18:33:11.482257 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.482234    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7e8b5de5-dnbwm"]
Sep 16 18:33:11.548353 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.548323    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:11.561475 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.561453    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qh5j\" (UniqueName: \"kubernetes.io/projected/877fef67-b5fa-429c-b894-34b1c1c4347a-kube-api-access-6qh5j\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:33:11.561475 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.561473    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/877fef67-b5fa-429c-b894-34b1c1c4347a-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:33:11.701316 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.701286    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:11.824530 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.824503    2562 generic.go:356] "Generic (PLEG): container finished" podID="877fef67-b5fa-429c-b894-34b1c1c4347a" containerID="ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a" exitCode=0
Sep 16 18:33:11.824659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.824536    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5" event={"ID":"877fef67-b5fa-429c-b894-34b1c1c4347a","Type":"ContainerDied","Data":"ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a"}
Sep 16 18:33:11.824659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.824548    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"
Sep 16 18:33:11.824659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.824602    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5" event={"ID":"877fef67-b5fa-429c-b894-34b1c1c4347a","Type":"ContainerDied","Data":"ccf8b4fa54f50ab34b0bea9f774da59a6c54e7a41566c766a5d04a8e0fd5dc55"}
Sep 16 18:33:11.824659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.824626    2562 scope.go:117] "RemoveContainer" containerID="ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a"
Sep 16 18:33:11.833978 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.833960    2562 scope.go:117] "RemoveContainer" containerID="ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a"
Sep 16 18:33:11.834234 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:33:11.834213    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a\": container with ID starting with ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a not found: ID does not exist" containerID="ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a"
Sep 16 18:33:11.834331 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.834240    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a"} err="failed to get container status \"ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a\": rpc error: code = NotFound desc = could not find container \"ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a\": container with ID starting with ea6f4bed8f7b0aae19ee814d1775f69661689c80681823cffb6917a28c30529a not found: ID does not exist"
Sep 16 18:33:11.847672 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.847648    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"]
Sep 16 18:33:11.854308 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:11.854287    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-7e8b5de5-84ddd8969c-hqxm5"]
Sep 16 18:33:12.285886 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:12.285864    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e085821-c192-4c2f-827a-8f8ff2a64dd3" path="/var/lib/kubelet/pods/4e085821-c192-4c2f-827a-8f8ff2a64dd3/volumes"
Sep 16 18:33:12.286182 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:12.286166    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="877fef67-b5fa-429c-b894-34b1c1c4347a" path="/var/lib/kubelet/pods/877fef67-b5fa-429c-b894-34b1c1c4347a/volumes"
Sep 16 18:33:12.402070 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:12.402045    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:12.486367 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:12.486339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:12.604720 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:12.604653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:12.733194 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:12.733173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:14.113782 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:14.113750    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:14.214296 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:14.214272    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:15.530871 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:15.530836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:16.672854 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:16.672819    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:17.946767 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:17.946735    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:20.293412 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:20.293386    2562 scope.go:117] "RemoveContainer" containerID="dff5f96678d65fe72830562c6de3ba2d74f95145a1e85decbe727aa74aecaca4"
Sep 16 18:33:22.836015 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:22.835978    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:22.957257 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:22.957223    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:23.083910 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:23.083876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:23.263870 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:23.263844    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:23.890261 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:23.890231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:28.074230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:28.074200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:30.514193 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:30.514158    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:31.823558 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:31.823507    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:33.446667 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:33.446621    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:33.956363 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:33.956331    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:35.427111 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:35.427080    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:36.929896 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:36.929865    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:38.202222 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:38.202191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:39.333338 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.333306    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"]
Sep 16 18:33:39.333729 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.333716    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="877fef67-b5fa-429c-b894-34b1c1c4347a" containerName="payload-processing"
Sep 16 18:33:39.333774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.333731    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="877fef67-b5fa-429c-b894-34b1c1c4347a" containerName="payload-processing"
Sep 16 18:33:39.333774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.333746    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e085821-c192-4c2f-827a-8f8ff2a64dd3" containerName="revoke-keys"
Sep 16 18:33:39.333774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.333751    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e085821-c192-4c2f-827a-8f8ff2a64dd3" containerName="revoke-keys"
Sep 16 18:33:39.333865 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.333807    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e085821-c192-4c2f-827a-8f8ff2a64dd3" containerName="revoke-keys"
Sep 16 18:33:39.333865 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.333815    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="877fef67-b5fa-429c-b894-34b1c1c4347a" containerName="payload-processing"
Sep 16 18:33:39.338435 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.338419    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.342127 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.342108    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-6942a4\"/\"premium-w4-6942a4-kserve-self-signed-certs\""
Sep 16 18:33:39.358015 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.357987    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"]
Sep 16 18:33:39.372155 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.372127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:39.481455 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.481426    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-home\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.481640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.481472    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-model-cache\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.481640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.481523    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldnvw\" (UniqueName: \"kubernetes.io/projected/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-kube-api-access-ldnvw\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.481640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.481541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tmp-dir\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.481836 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.481655    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tls-certs\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.481836 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.481702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-dshm\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.582431 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.582407    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tls-certs\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.582582 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.582448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-dshm\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.582582 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.582488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-home\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.582582 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.582523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-model-cache\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.582582 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.582553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldnvw\" (UniqueName: \"kubernetes.io/projected/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-kube-api-access-ldnvw\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.582801 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.582578    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tmp-dir\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.582994 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.582967    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tmp-dir\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.583095 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.583011    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-home\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.583095 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.583073    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-model-cache\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.584724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.584683    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-dshm\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.585051 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.585036    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tls-certs\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.598134 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.598113    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldnvw\" (UniqueName: \"kubernetes.io/projected/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-kube-api-access-ldnvw\") pod \"premium-w4-6942a4-kserve-6575bcbbcd-lqhwb\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") " pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.646833 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.646814    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:39.938662 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.938642    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"]
Sep 16 18:33:39.940815 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:33:39.940792    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8af2f79f_738b_40d4_a4f4_eff5f2a1b4df.slice/crio-51d6ab76440b4621030727793cc34ac011639a66ae307da2b1e3d2a68e218940 WatchSource:0}: Error finding container 51d6ab76440b4621030727793cc34ac011639a66ae307da2b1e3d2a68e218940: Status 404 returned error can't find the container with id 51d6ab76440b4621030727793cc34ac011639a66ae307da2b1e3d2a68e218940
Sep 16 18:33:39.969518 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:39.969489    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb" event={"ID":"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df","Type":"ContainerStarted","Data":"51d6ab76440b4621030727793cc34ac011639a66ae307da2b1e3d2a68e218940"}
Sep 16 18:33:40.976571 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:40.976538    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb" event={"ID":"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df","Type":"ContainerStarted","Data":"e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944"}
Sep 16 18:33:41.016508 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:41.016464    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb" podStartSLOduration=2.016450535 podStartE2EDuration="2.016450535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:33:41.012973162 +0000 UTC m=+1041.252761915" watchObservedRunningTime="2026-09-16 18:33:41.016450535 +0000 UTC m=+1041.256239286"
Sep 16 18:33:42.700343 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:42.700305    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:42.770702 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:42.770672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:49.647386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:49.647342    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:49.647386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:49.647398    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:49.659239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:49.659217    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:49.868643 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:49.868621    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:50.030219 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:50.030202    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:33:56.312321 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:56.312279    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:56.384246 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:56.384210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:56.478359 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:56.478332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:56.575958 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:56.575885    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:57.971873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:33:57.971846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:07.023060 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.023026    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"]
Sep 16 18:34:07.026526 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.026512    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.031851 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.031834    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-2c3173\"/\"openshift-service-ca.crt\""
Sep 16 18:34:07.032990 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.032971    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-2c3173\"/\"kube-root-ca.crt\""
Sep 16 18:34:07.033077 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.032980    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-2c3173\"/\"explicit-model-2c3173-kserve-self-signed-certs\""
Sep 16 18:34:07.033077 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.032980    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-2c3173\"/\"default-dockercfg-9pg64\""
Sep 16 18:34:07.039929 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.039906    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"]
Sep 16 18:34:07.049885 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.049863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:07.195635 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.195616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-dshm\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.195753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.195663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-home\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.195753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.195686    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tls-certs\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.195753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.195712    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-model-cache\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.195753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.195734    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tldvs\" (UniqueName: \"kubernetes.io/projected/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-kube-api-access-tldvs\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.195944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.195808    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tmp-dir\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.297193 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.297140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-home\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.297193 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.297167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tls-certs\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.297193 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.297183    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-model-cache\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.297410 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.297203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tldvs\" (UniqueName: \"kubernetes.io/projected/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-kube-api-access-tldvs\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.297410 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.297235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tmp-dir\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.297410 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.297279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-dshm\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.297560 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.297538    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-home\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.297655 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.297585    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-model-cache\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.297700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.297685    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tmp-dir\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.299524 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.299500    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-dshm\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.299785 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.299768    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tls-certs\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.307407 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.307381    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tldvs\" (UniqueName: \"kubernetes.io/projected/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-kube-api-access-tldvs\") pod \"explicit-model-2c3173-kserve-cb6b45b8c-55pd5\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") " pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.336775 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.336759    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:07.612695 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:07.612669    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"]
Sep 16 18:34:07.614030 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:34:07.614001    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b7e0202_c551_4da6_b5ea_6b33ee230a7a.slice/crio-bd80b51d013860bf51d78f59b2f20425012158aa2ec9ea06bb41d8f9265c3d46 WatchSource:0}: Error finding container bd80b51d013860bf51d78f59b2f20425012158aa2ec9ea06bb41d8f9265c3d46: Status 404 returned error can't find the container with id bd80b51d013860bf51d78f59b2f20425012158aa2ec9ea06bb41d8f9265c3d46
Sep 16 18:34:08.109212 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:08.109182    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5" event={"ID":"1b7e0202-c551-4da6-b5ea-6b33ee230a7a","Type":"ContainerStarted","Data":"c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13"}
Sep 16 18:34:08.109620 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:08.109218    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5" event={"ID":"1b7e0202-c551-4da6-b5ea-6b33ee230a7a","Type":"ContainerStarted","Data":"bd80b51d013860bf51d78f59b2f20425012158aa2ec9ea06bb41d8f9265c3d46"}
Sep 16 18:34:08.133268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:08.133226    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5" podStartSLOduration=2.133212345 podStartE2EDuration="2.133212345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:08.132446801 +0000 UTC m=+1068.372235563" watchObservedRunningTime="2026-09-16 18:34:08.133212345 +0000 UTC m=+1068.373001096"
Sep 16 18:34:12.431278 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:12.431208    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:17.337356 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:17.337322    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:17.337859 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:17.337369    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:17.349419 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:17.349398    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:18.179531 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:18.179507    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:18.596981 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:18.596900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:18.708609 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:18.708565    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:18.800034 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:18.800007    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:29.352153 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:29.352116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:30.127415 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:30.127386    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:36.613375 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:36.613345    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:36.726712 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:36.726680    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:36.815322 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:36.815296    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:36.920473 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:36.920447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:37.728814 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:37.728786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:37.843475 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:37.843417    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:39.352044 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:39.352015    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:40.368878 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:40.368849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:40.500000 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:40.499973    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:41.016544 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.016516    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"]
Sep 16 18:34:41.019733 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.019718    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"
Sep 16 18:34:41.023282 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.023262    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:34:41.023492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.023475    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:34:41.055245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.055224    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"]
Sep 16 18:34:41.078226 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.078195    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n\" (UID: \"957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"
Sep 16 18:34:41.179605 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.179567    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n\" (UID: \"957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"
Sep 16 18:34:41.180075 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.180057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n\" (UID: \"957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"
Sep 16 18:34:41.329667 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.329615    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"
Sep 16 18:34:41.447872 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.447841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:41.533838 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.533814    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:41.609527 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:41.609504    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"]
Sep 16 18:34:41.610940 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:34:41.610915    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod957b1b0d_f9b6_47b4_86f1_d524cb1b6d9d.slice/crio-d33846d115d67d5fc42a2ba4b9c6539984c8eeea065809b52f8e5f42264e0fc9 WatchSource:0}: Error finding container d33846d115d67d5fc42a2ba4b9c6539984c8eeea065809b52f8e5f42264e0fc9: Status 404 returned error can't find the container with id d33846d115d67d5fc42a2ba4b9c6539984c8eeea065809b52f8e5f42264e0fc9
Sep 16 18:34:42.081090 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:42.081058    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:42.296016 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:42.295985    2562 generic.go:356] "Generic (PLEG): container finished" podID="957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d" containerID="e4efdba3fd57935981dc5f67e321a7636fdd2cbe295bd58276323efaac40962f" exitCode=0
Sep 16 18:34:42.296111 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:42.296074    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n" event={"ID":"957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d","Type":"ContainerDied","Data":"e4efdba3fd57935981dc5f67e321a7636fdd2cbe295bd58276323efaac40962f"}
Sep 16 18:34:42.296111 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:42.296107    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n" event={"ID":"957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d","Type":"ContainerStarted","Data":"d33846d115d67d5fc42a2ba4b9c6539984c8eeea065809b52f8e5f42264e0fc9"}
Sep 16 18:34:43.429305 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:43.429280    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"
Sep 16 18:34:43.500851 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:43.500821    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d-maas-api-service-ca\") pod \"957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d\" (UID: \"957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d\") "
Sep 16 18:34:43.501125 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:43.501103    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d" (UID: "957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:34:43.506222 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:43.506197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:43.601816 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:43.601762    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:34:44.307567 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:44.307537    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"
Sep 16 18:34:44.307705 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:44.307583    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n" event={"ID":"957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d","Type":"ContainerDied","Data":"d33846d115d67d5fc42a2ba4b9c6539984c8eeea065809b52f8e5f42264e0fc9"}
Sep 16 18:34:44.307705 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:44.307649    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d33846d115d67d5fc42a2ba4b9c6539984c8eeea065809b52f8e5f42264e0fc9"
Sep 16 18:34:44.513338 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:44.513304    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"]
Sep 16 18:34:44.550133 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:44.550062    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:44.927788 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:44.927759    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:46.520435 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:46.520401    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:48.082794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:48.082757    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:49.561909 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:49.561880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:49.801305 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:49.801277    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:49.916374 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:49.916348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:50.563038 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:50.563011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:52.040579 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:52.040545    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:52.148726 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:52.148699    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:52.233221 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:52.233194    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:52.335553 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:52.335485    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:52.440024 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:52.439998    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"]
Sep 16 18:34:52.442322 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:52.442303    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2c3173-vmz5n"]
Sep 16 18:34:52.445798 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:52.445771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:52.802900 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:52.802868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:52.869394 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:52.869369    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:53.833409 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:53.833375    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:54.286155 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:54.286125    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d" path="/var/lib/kubelet/pods/957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d/volumes"
Sep 16 18:34:54.797334 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:54.797308    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:55.732441 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:55.732406    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:56.880294 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:56.880264    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:57.888947 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:57.888912    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:58.227771 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:58.227742    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:58.393630 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:58.393577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:58.773791 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:58.773761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:59.181315 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.181284    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:59.358164 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.358133    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:59.532083 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.532019    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5" podUID="1b7e0202-c551-4da6-b5ea-6b33ee230a7a" containerName="main" containerID="cri-o://c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13" gracePeriod=45
Sep 16 18:34:59.779115 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.779094    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:34:59.936094 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.936071    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tldvs\" (UniqueName: \"kubernetes.io/projected/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-kube-api-access-tldvs\") pod \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") "
Sep 16 18:34:59.936230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.936120    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tmp-dir\") pod \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") "
Sep 16 18:34:59.936230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.936149    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-home\") pod \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") "
Sep 16 18:34:59.936230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.936168    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-dshm\") pod \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") "
Sep 16 18:34:59.936230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.936186    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tls-certs\") pod \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") "
Sep 16 18:34:59.936230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.936214    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-model-cache\") pod \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\" (UID: \"1b7e0202-c551-4da6-b5ea-6b33ee230a7a\") "
Sep 16 18:34:59.936482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.936339    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1b7e0202-c551-4da6-b5ea-6b33ee230a7a" (UID: "1b7e0202-c551-4da6-b5ea-6b33ee230a7a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:59.936482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.936376    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-home" (OuterVolumeSpecName: "home") pod "1b7e0202-c551-4da6-b5ea-6b33ee230a7a" (UID: "1b7e0202-c551-4da6-b5ea-6b33ee230a7a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:59.936611 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.936563    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-model-cache" (OuterVolumeSpecName: "model-cache") pod "1b7e0202-c551-4da6-b5ea-6b33ee230a7a" (UID: "1b7e0202-c551-4da6-b5ea-6b33ee230a7a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:59.936704 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.936679    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:34:59.936704 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.936704    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:34:59.938379 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.938352    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-dshm" (OuterVolumeSpecName: "dshm") pod "1b7e0202-c551-4da6-b5ea-6b33ee230a7a" (UID: "1b7e0202-c551-4da6-b5ea-6b33ee230a7a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:34:59.938488 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.938421    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-kube-api-access-tldvs" (OuterVolumeSpecName: "kube-api-access-tldvs") pod "1b7e0202-c551-4da6-b5ea-6b33ee230a7a" (UID: "1b7e0202-c551-4da6-b5ea-6b33ee230a7a"). InnerVolumeSpecName "kube-api-access-tldvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:34:59.938488 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:34:59.938418    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1b7e0202-c551-4da6-b5ea-6b33ee230a7a" (UID: "1b7e0202-c551-4da6-b5ea-6b33ee230a7a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:35:00.037634 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.037580    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tldvs\" (UniqueName: \"kubernetes.io/projected/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-kube-api-access-tldvs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:35:00.037634 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.037628    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:35:00.037752 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.037638    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:35:00.037752 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.037647    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b7e0202-c551-4da6-b5ea-6b33ee230a7a-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:35:00.389572 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.389535    2562 generic.go:356] "Generic (PLEG): container finished" podID="1b7e0202-c551-4da6-b5ea-6b33ee230a7a" containerID="c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13" exitCode=0
Sep 16 18:35:00.389922 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.389574    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"
Sep 16 18:35:00.389922 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.389617    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5" event={"ID":"1b7e0202-c551-4da6-b5ea-6b33ee230a7a","Type":"ContainerDied","Data":"c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13"}
Sep 16 18:35:00.389922 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.389654    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5" event={"ID":"1b7e0202-c551-4da6-b5ea-6b33ee230a7a","Type":"ContainerDied","Data":"bd80b51d013860bf51d78f59b2f20425012158aa2ec9ea06bb41d8f9265c3d46"}
Sep 16 18:35:00.389922 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.389670    2562 scope.go:117] "RemoveContainer" containerID="c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13"
Sep 16 18:35:00.398265 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.398252    2562 scope.go:117] "RemoveContainer" containerID="c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13"
Sep 16 18:35:00.398565 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:35:00.398548    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13\": container with ID starting with c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13 not found: ID does not exist" containerID="c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13"
Sep 16 18:35:00.398636 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.398578    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13"} err="failed to get container status \"c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13\": rpc error: code = NotFound desc = could not find container \"c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13\": container with ID starting with c74899e2fe2cccb1e68d91591095e6d5c378a3cdcb90d776f0914df2a4bb0f13 not found: ID does not exist"
Sep 16 18:35:00.427413 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.427390    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"]
Sep 16 18:35:00.443574 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:00.443555    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-2c3173/explicit-model-2c3173-kserve-cb6b45b8c-55pd5"]
Sep 16 18:35:02.286057 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:02.286023    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b7e0202-c551-4da6-b5ea-6b33ee230a7a" path="/var/lib/kubelet/pods/1b7e0202-c551-4da6-b5ea-6b33ee230a7a/volumes"
Sep 16 18:35:05.646065 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:05.646027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:05.914838 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:05.914803    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:06.307171 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:06.307100    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:06.798560 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:06.798527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:06.997840 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:06.997807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:07.513343 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:07.513312    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:07.852025 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:07.851946    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:08.036007 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:08.035981    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:08.156565 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:08.156509    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:08.274934 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:08.274896    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:08.415276 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:08.415252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:09.023862 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:09.023834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:10.334232 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:10.334190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:11.586771 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:11.586718    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:12.529417 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:12.529384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:13.660808 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:13.660776    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:13.749958 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:13.749913    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:13.894018 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:13.893992    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:14.237107 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.237081    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:14.338244 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.338214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:14.367582 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.367557    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"]
Sep 16 18:35:14.368010 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.367995    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d" containerName="revoke-keys"
Sep 16 18:35:14.368058 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.368013    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d" containerName="revoke-keys"
Sep 16 18:35:14.368058 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.368029    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b7e0202-c551-4da6-b5ea-6b33ee230a7a" containerName="main"
Sep 16 18:35:14.368058 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.368035    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b7e0202-c551-4da6-b5ea-6b33ee230a7a" containerName="main"
Sep 16 18:35:14.368189 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.368100    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="957b1b0d-f9b6-47b4-86f1-d524cb1b6d9d" containerName="revoke-keys"
Sep 16 18:35:14.368189 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.368109    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b7e0202-c551-4da6-b5ea-6b33ee230a7a" containerName="main"
Sep 16 18:35:14.372630 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.372613    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:14.375032 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.375014    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-d485ed3e\""
Sep 16 18:35:14.386492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.386471    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"]
Sep 16 18:35:14.546756 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.546696    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac906afe-96a5-4096-8490-ce118efc275d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh\" (UID: \"ac906afe-96a5-4096-8490-ce118efc275d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:14.546756 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.546728    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktmhk\" (UniqueName: \"kubernetes.io/projected/ac906afe-96a5-4096-8490-ce118efc275d-kube-api-access-ktmhk\") pod \"payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh\" (UID: \"ac906afe-96a5-4096-8490-ce118efc275d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:14.647804 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.647778    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac906afe-96a5-4096-8490-ce118efc275d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh\" (UID: \"ac906afe-96a5-4096-8490-ce118efc275d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:14.647931 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.647812    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktmhk\" (UniqueName: \"kubernetes.io/projected/ac906afe-96a5-4096-8490-ce118efc275d-kube-api-access-ktmhk\") pod \"payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh\" (UID: \"ac906afe-96a5-4096-8490-ce118efc275d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:14.648461 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.648441    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac906afe-96a5-4096-8490-ce118efc275d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh\" (UID: \"ac906afe-96a5-4096-8490-ce118efc275d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:14.658875 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.658848    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktmhk\" (UniqueName: \"kubernetes.io/projected/ac906afe-96a5-4096-8490-ce118efc275d-kube-api-access-ktmhk\") pod \"payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh\" (UID: \"ac906afe-96a5-4096-8490-ce118efc275d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:14.683679 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.683660    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:14.958350 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:14.958321    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"]
Sep 16 18:35:14.958856 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:35:14.958835    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac906afe_96a5_4096_8490_ce118efc275d.slice/crio-f32da1cb6626d9383c782f92be5990164c2345c52be2342182758826378d25d8 WatchSource:0}: Error finding container f32da1cb6626d9383c782f92be5990164c2345c52be2342182758826378d25d8: Status 404 returned error can't find the container with id f32da1cb6626d9383c782f92be5990164c2345c52be2342182758826378d25d8
Sep 16 18:35:15.471319 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:15.471278    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh" event={"ID":"ac906afe-96a5-4096-8490-ce118efc275d","Type":"ContainerStarted","Data":"d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea"}
Sep 16 18:35:15.471468 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:15.471330    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh" event={"ID":"ac906afe-96a5-4096-8490-ce118efc275d","Type":"ContainerStarted","Data":"f32da1cb6626d9383c782f92be5990164c2345c52be2342182758826378d25d8"}
Sep 16 18:35:15.471468 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:15.471352    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:15.495584 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:15.495543    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh" podStartSLOduration=1.495528764 podStartE2EDuration="1.495528764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:15.491274958 +0000 UTC m=+1135.731063709" watchObservedRunningTime="2026-09-16 18:35:15.495528764 +0000 UTC m=+1135.735317566"
Sep 16 18:35:26.477780 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:26.477754    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:36.086513 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.086477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:36.414987 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.414955    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"]
Sep 16 18:35:36.420364 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.420343    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.423177 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.423158    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-6942a4\"/\"e2e-conflict-a-15e17679-kserve-self-signed-certs\""
Sep 16 18:35:36.432335 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.432310    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"]
Sep 16 18:35:36.527199 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.527170    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-model-cache\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.527318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.527206    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tmp-dir\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.527318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.527241    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl8br\" (UniqueName: \"kubernetes.io/projected/96c07cc6-17d9-40ce-9c4a-ea90783a6363-kube-api-access-hl8br\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.527407 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.527319    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-dshm\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.527407 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.527383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tls-certs\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.527471 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.527428    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-home\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.598734 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.598682    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:36.628046 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.628023    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tls-certs\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.628153 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.628060    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-home\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.628153 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.628099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-model-cache\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.628222 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.628207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tmp-dir\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.628267 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.628251    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl8br\" (UniqueName: \"kubernetes.io/projected/96c07cc6-17d9-40ce-9c4a-ea90783a6363-kube-api-access-hl8br\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.628365 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.628345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-dshm\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.628504 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.628450    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-home\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.628504 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.628464    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-model-cache\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.628655 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.628545    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tmp-dir\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.630505 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.630486    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-dshm\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.630710 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.630690    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tls-certs\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.637119 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.637097    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl8br\" (UniqueName: \"kubernetes.io/projected/96c07cc6-17d9-40ce-9c4a-ea90783a6363-kube-api-access-hl8br\") pod \"e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:36.728810 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:36.728756    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:37.002135 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:37.002108    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"]
Sep 16 18:35:37.005020 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:35:37.004991    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96c07cc6_17d9_40ce_9c4a_ea90783a6363.slice/crio-c3800d5a66c43f72c35fadcc27ab6b2ff7f155fb5d171b23b0482e7aff6155e2 WatchSource:0}: Error finding container c3800d5a66c43f72c35fadcc27ab6b2ff7f155fb5d171b23b0482e7aff6155e2: Status 404 returned error can't find the container with id c3800d5a66c43f72c35fadcc27ab6b2ff7f155fb5d171b23b0482e7aff6155e2
Sep 16 18:35:37.025676 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:37.025652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:37.590767 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:37.590737    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns" event={"ID":"96c07cc6-17d9-40ce-9c4a-ea90783a6363","Type":"ContainerStarted","Data":"3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725"}
Sep 16 18:35:37.590767 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:37.590769    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns" event={"ID":"96c07cc6-17d9-40ce-9c4a-ea90783a6363","Type":"ContainerStarted","Data":"c3800d5a66c43f72c35fadcc27ab6b2ff7f155fb5d171b23b0482e7aff6155e2"}
Sep 16 18:35:37.610035 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:37.609995    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns" podStartSLOduration=1.6099807259999999 podStartE2EDuration="1.609980726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:37.608997905 +0000 UTC m=+1157.848786658" watchObservedRunningTime="2026-09-16 18:35:37.609980726 +0000 UTC m=+1157.849769524"
Sep 16 18:35:37.764176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:37.764150    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:38.179618 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:38.179572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:38.799457 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:38.799425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:38.899766 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:38.899743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:39.009938 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:39.009914    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:39.242552 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:39.242526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:39.993825 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:39.993788    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:40.942734 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:40.942700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:46.111443 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.111364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:46.235193 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.235166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:46.491357 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.491324    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"]
Sep 16 18:35:46.495090 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.495070    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"
Sep 16 18:35:46.497730 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.497707    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:35:46.497933 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.497920    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:35:46.503670 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.503651    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"]
Sep 16 18:35:46.609767 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.609739    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2059221-040d-4155-872a-73267518f043-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d485ed3e-clftm\" (UID: \"c2059221-040d-4155-872a-73267518f043\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"
Sep 16 18:35:46.710665 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.710641    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2059221-040d-4155-872a-73267518f043-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d485ed3e-clftm\" (UID: \"c2059221-040d-4155-872a-73267518f043\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"
Sep 16 18:35:46.711258 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.711240    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2059221-040d-4155-872a-73267518f043-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d485ed3e-clftm\" (UID: \"c2059221-040d-4155-872a-73267518f043\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"
Sep 16 18:35:46.729020 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.728998    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:46.729109 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.729029    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:46.741029 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.741013    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:46.807918 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:46.807870    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"
Sep 16 18:35:47.086602 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:47.086562    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"]
Sep 16 18:35:47.088648 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:35:47.088621    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2059221_040d_4155_872a_73267518f043.slice/crio-d43300b2cdd82110f8a31afa91857348fd901611157c129194d70dfdec310660 WatchSource:0}: Error finding container d43300b2cdd82110f8a31afa91857348fd901611157c129194d70dfdec310660: Status 404 returned error can't find the container with id d43300b2cdd82110f8a31afa91857348fd901611157c129194d70dfdec310660
Sep 16 18:35:47.645974 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:47.645939    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm" event={"ID":"c2059221-040d-4155-872a-73267518f043","Type":"ContainerStarted","Data":"2b315c3ef3b3e3f4d202f485b0160968650ecffb393246628c538a54f83d7074"}
Sep 16 18:35:47.646373 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:47.645999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm" event={"ID":"c2059221-040d-4155-872a-73267518f043","Type":"ContainerStarted","Data":"d43300b2cdd82110f8a31afa91857348fd901611157c129194d70dfdec310660"}
Sep 16 18:35:47.656776 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:47.656755    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:35:47.685047 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:47.685025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:47.701811 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:47.701767    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm" podStartSLOduration=1.701754483 podStartE2EDuration="1.701754483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:47.701287042 +0000 UTC m=+1167.941075795" watchObservedRunningTime="2026-09-16 18:35:47.701754483 +0000 UTC m=+1167.941543286"
Sep 16 18:35:48.652809 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:48.652780    2562 generic.go:356] "Generic (PLEG): container finished" podID="c2059221-040d-4155-872a-73267518f043" containerID="2b315c3ef3b3e3f4d202f485b0160968650ecffb393246628c538a54f83d7074" exitCode=0
Sep 16 18:35:48.653149 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:48.652845    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm" event={"ID":"c2059221-040d-4155-872a-73267518f043","Type":"ContainerDied","Data":"2b315c3ef3b3e3f4d202f485b0160968650ecffb393246628c538a54f83d7074"}
Sep 16 18:35:48.775399 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:48.775371    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:48.926862 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:48.926838    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:49.670130 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:49.670101    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:49.771003 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:49.770968    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:49.794034 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:49.794015    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"
Sep 16 18:35:49.941824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:49.941799    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2059221-040d-4155-872a-73267518f043-maas-api-service-ca\") pod \"c2059221-040d-4155-872a-73267518f043\" (UID: \"c2059221-040d-4155-872a-73267518f043\") "
Sep 16 18:35:49.942088 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:49.942069    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2059221-040d-4155-872a-73267518f043-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c2059221-040d-4155-872a-73267518f043" (UID: "c2059221-040d-4155-872a-73267518f043"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:35:50.042840 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:50.042816    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2059221-040d-4155-872a-73267518f043-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:35:50.628664 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:50.628559    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:50.665881 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:50.665850    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm" event={"ID":"c2059221-040d-4155-872a-73267518f043","Type":"ContainerDied","Data":"d43300b2cdd82110f8a31afa91857348fd901611157c129194d70dfdec310660"}
Sep 16 18:35:50.666024 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:50.665890    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d43300b2cdd82110f8a31afa91857348fd901611157c129194d70dfdec310660"
Sep 16 18:35:50.666024 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:50.665855    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"
Sep 16 18:35:51.412209 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:51.412178    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:52.401361 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:52.401325    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:57.182397 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.182367    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:57.222401 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.222371    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"]
Sep 16 18:35:57.222637 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.222615    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh" podUID="ac906afe-96a5-4096-8490-ce118efc275d" containerName="payload-pre-processing" containerID="cri-o://d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea" gracePeriod=30
Sep 16 18:35:57.285283 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.285246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:57.384698 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.384653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:57.429910 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.429882    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"]
Sep 16 18:35:57.435133 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.435077    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d485ed3e-clftm"]
Sep 16 18:35:57.471682 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.471660    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:57.538371 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.538333    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:57.608748 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.608727    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac906afe-96a5-4096-8490-ce118efc275d-plugins-config-volume\") pod \"ac906afe-96a5-4096-8490-ce118efc275d\" (UID: \"ac906afe-96a5-4096-8490-ce118efc275d\") "
Sep 16 18:35:57.608840 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.608788    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktmhk\" (UniqueName: \"kubernetes.io/projected/ac906afe-96a5-4096-8490-ce118efc275d-kube-api-access-ktmhk\") pod \"ac906afe-96a5-4096-8490-ce118efc275d\" (UID: \"ac906afe-96a5-4096-8490-ce118efc275d\") "
Sep 16 18:35:57.609064 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.609044    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac906afe-96a5-4096-8490-ce118efc275d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac906afe-96a5-4096-8490-ce118efc275d" (UID: "ac906afe-96a5-4096-8490-ce118efc275d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:35:57.610951 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.610928    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac906afe-96a5-4096-8490-ce118efc275d-kube-api-access-ktmhk" (OuterVolumeSpecName: "kube-api-access-ktmhk") pod "ac906afe-96a5-4096-8490-ce118efc275d" (UID: "ac906afe-96a5-4096-8490-ce118efc275d"). InnerVolumeSpecName "kube-api-access-ktmhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:35:57.704671 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.704610    2562 generic.go:356] "Generic (PLEG): container finished" podID="ac906afe-96a5-4096-8490-ce118efc275d" containerID="d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea" exitCode=0
Sep 16 18:35:57.704671 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.704644    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"
Sep 16 18:35:57.704795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.704693    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh" event={"ID":"ac906afe-96a5-4096-8490-ce118efc275d","Type":"ContainerDied","Data":"d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea"}
Sep 16 18:35:57.704795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.704737    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh" event={"ID":"ac906afe-96a5-4096-8490-ce118efc275d","Type":"ContainerDied","Data":"f32da1cb6626d9383c782f92be5990164c2345c52be2342182758826378d25d8"}
Sep 16 18:35:57.704795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.704758    2562 scope.go:117] "RemoveContainer" containerID="d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea"
Sep 16 18:35:57.709534 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.709516    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktmhk\" (UniqueName: \"kubernetes.io/projected/ac906afe-96a5-4096-8490-ce118efc275d-kube-api-access-ktmhk\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:35:57.709534 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.709536    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac906afe-96a5-4096-8490-ce118efc275d-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:35:57.714079 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.714064    2562 scope.go:117] "RemoveContainer" containerID="d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea"
Sep 16 18:35:57.714380 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:35:57.714363    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea\": container with ID starting with d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea not found: ID does not exist" containerID="d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea"
Sep 16 18:35:57.714436 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.714387    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea"} err="failed to get container status \"d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea\": rpc error: code = NotFound desc = could not find container \"d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea\": container with ID starting with d0030aecd46d399bbfa047a821a786a0757bbe2f9315a995f00b88ba53ec42ea not found: ID does not exist"
Sep 16 18:35:57.729230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.729210    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"]
Sep 16 18:35:57.731154 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.731135    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d485ed3e-6f794c4fd6-6jrmh"]
Sep 16 18:35:57.793380 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.793356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:57.908158 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:57.908136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:58.286526 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:58.286504    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac906afe-96a5-4096-8490-ce118efc275d" path="/var/lib/kubelet/pods/ac906afe-96a5-4096-8490-ce118efc275d/volumes"
Sep 16 18:35:58.286832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:58.286819    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2059221-040d-4155-872a-73267518f043" path="/var/lib/kubelet/pods/c2059221-040d-4155-872a-73267518f043/volumes"
Sep 16 18:35:58.653177 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:58.653120    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:59.548904 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:35:59.548875    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:00.329314 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:00.329284    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:01.250460 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:01.250428    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:02.215134 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:02.215097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:04.262698 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:04.262669    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:04.366521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:04.366497    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:04.646332 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:04.646249    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:04.986238 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:04.986209    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:05.078697 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:05.078674    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:05.239758 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:05.239703    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:05.392103 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:05.392076    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:10.814652 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.814620    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"]
Sep 16 18:36:10.815069 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.815053    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac906afe-96a5-4096-8490-ce118efc275d" containerName="payload-pre-processing"
Sep 16 18:36:10.815125 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.815072    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac906afe-96a5-4096-8490-ce118efc275d" containerName="payload-pre-processing"
Sep 16 18:36:10.815125 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.815093    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2059221-040d-4155-872a-73267518f043" containerName="revoke-keys"
Sep 16 18:36:10.815125 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.815098    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2059221-040d-4155-872a-73267518f043" containerName="revoke-keys"
Sep 16 18:36:10.815223 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.815174    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac906afe-96a5-4096-8490-ce118efc275d" containerName="payload-pre-processing"
Sep 16 18:36:10.815223 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.815183    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2059221-040d-4155-872a-73267518f043" containerName="revoke-keys"
Sep 16 18:36:10.819605 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.819573    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:10.822546 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.822530    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a24d09\"/\"distinct-w0-a24d09-kserve-self-signed-certs\""
Sep 16 18:36:10.841203 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.841179    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"]
Sep 16 18:36:10.847013 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.846975    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:10.915486 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.915464    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tls-certs\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:10.915643 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.915517    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-model-cache\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:10.915643 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.915582    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-dshm\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:10.915779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.915693    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-home\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:10.915779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.915728    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwqvz\" (UniqueName: \"kubernetes.io/projected/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-kube-api-access-lwqvz\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:10.915779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:10.915762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tmp-dir\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.016886 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.016862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-home\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.016990 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.016895    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwqvz\" (UniqueName: \"kubernetes.io/projected/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-kube-api-access-lwqvz\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.016990 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.016914    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tmp-dir\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.016990 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.016950    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tls-certs\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.017146 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.017001    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-model-cache\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.017146 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.017049    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-dshm\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.017308 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.017290    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-home\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.017308 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.017301    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tmp-dir\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.017389 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.017369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-model-cache\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.019360 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.019332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-dshm\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.019577 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.019560    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tls-certs\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.028351 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.028332    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwqvz\" (UniqueName: \"kubernetes.io/projected/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-kube-api-access-lwqvz\") pod \"distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") " pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.128689 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.128634    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:11.413042 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.412934    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"]
Sep 16 18:36:11.415247 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:36:11.415222    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9e8c23c_fd1c_4e01_8d74_d9c37584de9e.slice/crio-25acad902fe41b2161f95fadfc077aa88b7174432620bddf5a8dae18d43b1ff9 WatchSource:0}: Error finding container 25acad902fe41b2161f95fadfc077aa88b7174432620bddf5a8dae18d43b1ff9: Status 404 returned error can't find the container with id 25acad902fe41b2161f95fadfc077aa88b7174432620bddf5a8dae18d43b1ff9
Sep 16 18:36:11.780442 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.780375    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx" event={"ID":"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e","Type":"ContainerStarted","Data":"50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661"}
Sep 16 18:36:11.780442 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.780413    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx" event={"ID":"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e","Type":"ContainerStarted","Data":"25acad902fe41b2161f95fadfc077aa88b7174432620bddf5a8dae18d43b1ff9"}
Sep 16 18:36:11.801975 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.801927    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx" podStartSLOduration=1.8019137349999998 podStartE2EDuration="1.801913735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:36:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:36:11.799108047 +0000 UTC m=+1192.038896800" watchObservedRunningTime="2026-09-16 18:36:11.801913735 +0000 UTC m=+1192.041702486"
Sep 16 18:36:11.991175 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:11.991148    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:12.092208 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:12.092149    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:12.189203 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:12.189183    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:12.366814 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:12.366745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:13.866888 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:13.866856    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:13.960813 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:13.960770    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:14.237070 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:14.237036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:14.339377 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:14.339350    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:15.131031 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:15.130994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:18.228549 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:18.228513    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:18.845805 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:18.845767    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:18.916038 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:18.916013    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:19.501775 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.501744    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:19.789110 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.789030    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"]
Sep 16 18:36:19.792844 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.792824    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.795750 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.795730    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-6942a4\"/\"e2e-conflict-b-15e17679-kserve-self-signed-certs\""
Sep 16 18:36:19.807256 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.807236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:19.826835 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.826813    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"]
Sep 16 18:36:19.894452 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.894430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tls-certs\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.894577 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.894469    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-model-cache\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.894669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.894572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-home\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.894669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.894662    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfgsr\" (UniqueName: \"kubernetes.io/projected/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-kube-api-access-hfgsr\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.894751 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.894698    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-dshm\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.894751 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.894722    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tmp-dir\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.926992 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.926952    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:19.995527 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.995501    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-home\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.995673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.995544    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfgsr\" (UniqueName: \"kubernetes.io/projected/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-kube-api-access-hfgsr\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.995673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.995584    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-dshm\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.995673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.995638    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tmp-dir\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.995673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.995669    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tls-certs\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.995889 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.995712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-model-cache\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.995961 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.995943    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-home\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.996089 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.996062    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tmp-dir\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.996145 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.996116    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-model-cache\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.998029 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.998007    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-dshm\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:19.998156 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:19.998140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tls-certs\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:20.008786 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:20.008764    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfgsr\" (UniqueName: \"kubernetes.io/projected/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-kube-api-access-hfgsr\") pod \"e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") " pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:20.102353 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:20.102301    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:20.362733 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:20.362693    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:20.393149 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:20.393116    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"]
Sep 16 18:36:20.394132 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:36:20.394110    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podedc06a7c_78f7_4ecf_9293_90d4ce5cbdf6.slice/crio-de35371cbe175b9f767028ee32b70e45af240a28ebfa2d356fad6358963f4e50 WatchSource:0}: Error finding container de35371cbe175b9f767028ee32b70e45af240a28ebfa2d356fad6358963f4e50: Status 404 returned error can't find the container with id de35371cbe175b9f767028ee32b70e45af240a28ebfa2d356fad6358963f4e50
Sep 16 18:36:20.724186 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:20.724159    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:20.832235 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:20.832203    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg" event={"ID":"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6","Type":"ContainerStarted","Data":"43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537"}
Sep 16 18:36:20.832383 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:20.832251    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg" event={"ID":"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6","Type":"ContainerStarted","Data":"de35371cbe175b9f767028ee32b70e45af240a28ebfa2d356fad6358963f4e50"}
Sep 16 18:36:20.891709 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:20.891660    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg" podStartSLOduration=1.891645881 podStartE2EDuration="1.891645881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:36:20.890370689 +0000 UTC m=+1201.130159443" watchObservedRunningTime="2026-09-16 18:36:20.891645881 +0000 UTC m=+1201.131434632"
Sep 16 18:36:21.129717 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:21.129656    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:21.129717 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:21.129698    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:21.141766 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:21.141740    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:21.847849 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:21.847824    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:36:23.281610 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:23.281561    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:23.930018 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:23.929979    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:24.031719 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:24.031690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:24.261003 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:24.260934    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:25.262209 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:25.262175    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:26.287894 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:26.287864    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:27.144858 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:27.144829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:28.330975 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:28.330942    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:28.422281 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:28.422254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:28.520498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:28.520473    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:28.898942 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:28.898911    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:28.997893 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:28.997860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:30.102496 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:30.102470    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:30.102878 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:30.102513    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:30.114332 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:30.114307    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:30.896541 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:30.896519    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:36:46.097724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.097692    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"]
Sep 16 18:36:46.102308 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.102285    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"
Sep 16 18:36:46.105021 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.105005    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:36:46.105231 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.105213    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:36:46.115801 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.115780    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"]
Sep 16 18:36:46.219828 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.219794    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e671378-9892-4e69-87ab-f62eaa074493-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps\" (UID: \"9e671378-9892-4e69-87ab-f62eaa074493\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"
Sep 16 18:36:46.320470 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.320445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e671378-9892-4e69-87ab-f62eaa074493-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps\" (UID: \"9e671378-9892-4e69-87ab-f62eaa074493\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"
Sep 16 18:36:46.321075 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.321058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e671378-9892-4e69-87ab-f62eaa074493-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps\" (UID: \"9e671378-9892-4e69-87ab-f62eaa074493\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"
Sep 16 18:36:46.413195 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.413177    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"
Sep 16 18:36:46.454623 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.454572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:46.686774 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.686749    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"]
Sep 16 18:36:46.688607 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:36:46.688554    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e671378_9892_4e69_87ab_f62eaa074493.slice/crio-0feef617b4171862273057f5ca2882c40044444a55f93d336a60d0ea061001ab WatchSource:0}: Error finding container 0feef617b4171862273057f5ca2882c40044444a55f93d336a60d0ea061001ab: Status 404 returned error can't find the container with id 0feef617b4171862273057f5ca2882c40044444a55f93d336a60d0ea061001ab
Sep 16 18:36:46.969778 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.969714    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps" event={"ID":"9e671378-9892-4e69-87ab-f62eaa074493","Type":"ContainerStarted","Data":"d4dbc0c62c4507d07b7de2ab8b68b6c21e3abf6639b73cb3d21aed70e33ee835"}
Sep 16 18:36:46.969778 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:46.969765    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps" event={"ID":"9e671378-9892-4e69-87ab-f62eaa074493","Type":"ContainerStarted","Data":"0feef617b4171862273057f5ca2882c40044444a55f93d336a60d0ea061001ab"}
Sep 16 18:36:47.569115 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:47.569089    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:47.976159 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:47.976135    2562 generic.go:356] "Generic (PLEG): container finished" podID="9e671378-9892-4e69-87ab-f62eaa074493" containerID="d4dbc0c62c4507d07b7de2ab8b68b6c21e3abf6639b73cb3d21aed70e33ee835" exitCode=0
Sep 16 18:36:47.976290 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:47.976201    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps" event={"ID":"9e671378-9892-4e69-87ab-f62eaa074493","Type":"ContainerDied","Data":"d4dbc0c62c4507d07b7de2ab8b68b6c21e3abf6639b73cb3d21aed70e33ee835"}
Sep 16 18:36:48.347944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:48.347853    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:48.939285 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:48.939250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:49.117134 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:49.117108    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"
Sep 16 18:36:49.184950 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:49.184920    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:49.252734 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:49.252673    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e671378-9892-4e69-87ab-f62eaa074493-maas-api-service-ca\") pod \"9e671378-9892-4e69-87ab-f62eaa074493\" (UID: \"9e671378-9892-4e69-87ab-f62eaa074493\") "
Sep 16 18:36:49.253051 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:49.253027    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e671378-9892-4e69-87ab-f62eaa074493-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e671378-9892-4e69-87ab-f62eaa074493" (UID: "9e671378-9892-4e69-87ab-f62eaa074493"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:36:49.253134 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:49.253119    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e671378-9892-4e69-87ab-f62eaa074493-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:36:49.305696 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:49.305670    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:49.612757 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:49.612663    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:49.988613 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:49.988575    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"
Sep 16 18:36:49.988969 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:49.988621    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps" event={"ID":"9e671378-9892-4e69-87ab-f62eaa074493","Type":"ContainerDied","Data":"0feef617b4171862273057f5ca2882c40044444a55f93d336a60d0ea061001ab"}
Sep 16 18:36:49.988969 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:49.988664    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0feef617b4171862273057f5ca2882c40044444a55f93d336a60d0ea061001ab"
Sep 16 18:36:50.100829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:50.100793    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:50.199318 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:50.199284    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:50.943516 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:50.943488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:51.849963 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:51.849933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:52.904896 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:52.904864    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:53.084413 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:53.084384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:53.180994 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:53.180967    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:53.933639 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:53.933608    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:54.758738 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:54.758701    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:54.851546 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:54.851515    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:55.061241 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:55.061102    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:55.266352 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:55.266322    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:55.361848 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:55.361780    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:55.708933 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:55.708898    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:56.704346 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:56.704317    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:57.621003 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:57.620971    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:57.821061 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:57.821033    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:58.611141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.611107    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:58.642762 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.642736    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"]
Sep 16 18:36:58.643107 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.642974    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27" podUID="9675c1ec-78db-4511-8621-ff9ab01f822a" containerName="payload-processing" containerID="cri-o://511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87" gracePeriod=30
Sep 16 18:36:58.647225 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.647199    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"]
Sep 16 18:36:58.647438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.647406    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b" podUID="06550cad-4e3a-415e-b698-ba10dd9b8902" containerName="payload-pre-processing" containerID="cri-o://48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1" gracePeriod=30
Sep 16 18:36:58.689274 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.689248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:58.791849 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.791802    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:58.890974 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.890892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:58.921142 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.921124    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:36:58.923118 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.923098    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"]
Sep 16 18:36:58.924465 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.924445    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:36:58.925532 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.925517    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0197e4-tg9ps"]
Sep 16 18:36:58.969852 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:58.969827    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:59.033537 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.033518    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06550cad-4e3a-415e-b698-ba10dd9b8902-plugins-config-volume\") pod \"06550cad-4e3a-415e-b698-ba10dd9b8902\" (UID: \"06550cad-4e3a-415e-b698-ba10dd9b8902\") "
Sep 16 18:36:59.033672 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.033576    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx225\" (UniqueName: \"kubernetes.io/projected/06550cad-4e3a-415e-b698-ba10dd9b8902-kube-api-access-cx225\") pod \"06550cad-4e3a-415e-b698-ba10dd9b8902\" (UID: \"06550cad-4e3a-415e-b698-ba10dd9b8902\") "
Sep 16 18:36:59.033720 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.033680    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zl6jv\" (UniqueName: \"kubernetes.io/projected/9675c1ec-78db-4511-8621-ff9ab01f822a-kube-api-access-zl6jv\") pod \"9675c1ec-78db-4511-8621-ff9ab01f822a\" (UID: \"9675c1ec-78db-4511-8621-ff9ab01f822a\") "
Sep 16 18:36:59.033720 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.033702    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9675c1ec-78db-4511-8621-ff9ab01f822a-plugins-config-volume\") pod \"9675c1ec-78db-4511-8621-ff9ab01f822a\" (UID: \"9675c1ec-78db-4511-8621-ff9ab01f822a\") "
Sep 16 18:36:59.034043 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.033954    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06550cad-4e3a-415e-b698-ba10dd9b8902-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "06550cad-4e3a-415e-b698-ba10dd9b8902" (UID: "06550cad-4e3a-415e-b698-ba10dd9b8902"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:36:59.034153 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.034099    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9675c1ec-78db-4511-8621-ff9ab01f822a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9675c1ec-78db-4511-8621-ff9ab01f822a" (UID: "9675c1ec-78db-4511-8621-ff9ab01f822a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:36:59.035965 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.035944    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9675c1ec-78db-4511-8621-ff9ab01f822a-kube-api-access-zl6jv" (OuterVolumeSpecName: "kube-api-access-zl6jv") pod "9675c1ec-78db-4511-8621-ff9ab01f822a" (UID: "9675c1ec-78db-4511-8621-ff9ab01f822a"). InnerVolumeSpecName "kube-api-access-zl6jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:36:59.036063 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.035986    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06550cad-4e3a-415e-b698-ba10dd9b8902-kube-api-access-cx225" (OuterVolumeSpecName: "kube-api-access-cx225") pod "06550cad-4e3a-415e-b698-ba10dd9b8902" (UID: "06550cad-4e3a-415e-b698-ba10dd9b8902"). InnerVolumeSpecName "kube-api-access-cx225". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:36:59.038176 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.038155    2562 generic.go:356] "Generic (PLEG): container finished" podID="06550cad-4e3a-415e-b698-ba10dd9b8902" containerID="48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1" exitCode=0
Sep 16 18:36:59.038273 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.038205    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"
Sep 16 18:36:59.038273 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.038230    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b" event={"ID":"06550cad-4e3a-415e-b698-ba10dd9b8902","Type":"ContainerDied","Data":"48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1"}
Sep 16 18:36:59.038391 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.038283    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b" event={"ID":"06550cad-4e3a-415e-b698-ba10dd9b8902","Type":"ContainerDied","Data":"9ce3a477484bd844efd1280b957281497a8c64d3d4d1b3d9a3931c043d5462eb"}
Sep 16 18:36:59.038391 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.038305    2562 scope.go:117] "RemoveContainer" containerID="48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1"
Sep 16 18:36:59.039993 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.039969    2562 generic.go:356] "Generic (PLEG): container finished" podID="9675c1ec-78db-4511-8621-ff9ab01f822a" containerID="511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87" exitCode=0
Sep 16 18:36:59.040113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.040000    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27" event={"ID":"9675c1ec-78db-4511-8621-ff9ab01f822a","Type":"ContainerDied","Data":"511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87"}
Sep 16 18:36:59.040113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.040022    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"
Sep 16 18:36:59.040113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.040043    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27" event={"ID":"9675c1ec-78db-4511-8621-ff9ab01f822a","Type":"ContainerDied","Data":"5460eef079c0f72f9a385988cbaae7c6dafcfbe97bec3c3b83a72a0d08455a9c"}
Sep 16 18:36:59.055831 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.055810    2562 scope.go:117] "RemoveContainer" containerID="48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1"
Sep 16 18:36:59.056375 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:36:59.056351    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1\": container with ID starting with 48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1 not found: ID does not exist" containerID="48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1"
Sep 16 18:36:59.056469 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.056385    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1"} err="failed to get container status \"48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1\": rpc error: code = NotFound desc = could not find container \"48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1\": container with ID starting with 48de90b133bc95f8a05c5c6ca1bceea1ffaf6c8b34ac73688f468ab10e2dfdc1 not found: ID does not exist"
Sep 16 18:36:59.056469 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.056404    2562 scope.go:117] "RemoveContainer" containerID="511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87"
Sep 16 18:36:59.056917 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.056893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:59.064501 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.064481    2562 scope.go:117] "RemoveContainer" containerID="511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87"
Sep 16 18:36:59.064788 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:36:59.064772    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87\": container with ID starting with 511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87 not found: ID does not exist" containerID="511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87"
Sep 16 18:36:59.064846 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.064806    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87"} err="failed to get container status \"511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87\": rpc error: code = NotFound desc = could not find container \"511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87\": container with ID starting with 511837a6a51f10d5d190657b1a213bf547b3e9e8ea05b4ef8b8a1e8557eceb87 not found: ID does not exist"
Sep 16 18:36:59.080474 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.080448    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"]
Sep 16 18:36:59.089748 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.089729    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-0197e4-5c8c5fb54b-6ch6b"]
Sep 16 18:36:59.108298 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.108273    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"]
Sep 16 18:36:59.112957 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.112937    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0197e4-67fb88fbc7-4wx27"]
Sep 16 18:36:59.134553 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.134530    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zl6jv\" (UniqueName: \"kubernetes.io/projected/9675c1ec-78db-4511-8621-ff9ab01f822a-kube-api-access-zl6jv\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:36:59.134553 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.134550    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9675c1ec-78db-4511-8621-ff9ab01f822a-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:36:59.134680 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.134559    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06550cad-4e3a-415e-b698-ba10dd9b8902-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:36:59.134680 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.134568    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx225\" (UniqueName: \"kubernetes.io/projected/06550cad-4e3a-415e-b698-ba10dd9b8902-kube-api-access-cx225\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:36:59.260508 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:36:59.260479    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:00.222909 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:00.222881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:00.284819 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:00.284793    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06550cad-4e3a-415e-b698-ba10dd9b8902" path="/var/lib/kubelet/pods/06550cad-4e3a-415e-b698-ba10dd9b8902/volumes"
Sep 16 18:37:00.285131 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:00.285119    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9675c1ec-78db-4511-8621-ff9ab01f822a" path="/var/lib/kubelet/pods/9675c1ec-78db-4511-8621-ff9ab01f822a/volumes"
Sep 16 18:37:00.285395 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:00.285385    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e671378-9892-4e69-87ab-f62eaa074493" path="/var/lib/kubelet/pods/9e671378-9892-4e69-87ab-f62eaa074493/volumes"
Sep 16 18:37:01.121275 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:01.121240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:02.062669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:02.062637    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:02.880517 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:02.880486    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:04.451713 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:04.451684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:04.550382 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:04.550330    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:05.820675 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:05.820652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:05.922138 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:05.922107    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:06.026028 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.025991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:06.091333 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.091254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:06.440569 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.440545    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"]
Sep 16 18:37:06.440953 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.440941    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06550cad-4e3a-415e-b698-ba10dd9b8902" containerName="payload-pre-processing"
Sep 16 18:37:06.440996 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.440955    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="06550cad-4e3a-415e-b698-ba10dd9b8902" containerName="payload-pre-processing"
Sep 16 18:37:06.440996 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.440967    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9675c1ec-78db-4511-8621-ff9ab01f822a" containerName="payload-processing"
Sep 16 18:37:06.440996 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.440973    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9675c1ec-78db-4511-8621-ff9ab01f822a" containerName="payload-processing"
Sep 16 18:37:06.440996 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.440985    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e671378-9892-4e69-87ab-f62eaa074493" containerName="revoke-keys"
Sep 16 18:37:06.440996 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.440992    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e671378-9892-4e69-87ab-f62eaa074493" containerName="revoke-keys"
Sep 16 18:37:06.441149 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.441055    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e671378-9892-4e69-87ab-f62eaa074493" containerName="revoke-keys"
Sep 16 18:37:06.441149 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.441061    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="06550cad-4e3a-415e-b698-ba10dd9b8902" containerName="payload-pre-processing"
Sep 16 18:37:06.441149 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.441069    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9675c1ec-78db-4511-8621-ff9ab01f822a" containerName="payload-processing"
Sep 16 18:37:06.445666 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.445649    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"
Sep 16 18:37:06.448255 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.448238    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:37:06.448397 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.448362    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:37:06.453271 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.453049    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"]
Sep 16 18:37:06.592478 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.592449    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc2cd6fc-1c17-4997-904d-74d3b6124709-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl\" (UID: \"fc2cd6fc-1c17-4997-904d-74d3b6124709\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"
Sep 16 18:37:06.693812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.693754    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc2cd6fc-1c17-4997-904d-74d3b6124709-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl\" (UID: \"fc2cd6fc-1c17-4997-904d-74d3b6124709\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"
Sep 16 18:37:06.694281 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.694262    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc2cd6fc-1c17-4997-904d-74d3b6124709-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl\" (UID: \"fc2cd6fc-1c17-4997-904d-74d3b6124709\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"
Sep 16 18:37:06.757232 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.757207    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"
Sep 16 18:37:06.923485 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:06.923448    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:07.055482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:07.055461    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"]
Sep 16 18:37:07.057647 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:37:07.057623    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc2cd6fc_1c17_4997_904d_74d3b6124709.slice/crio-b33ec2b9e6c50fa9dad3f0d37c7c333bd4a8cf435f9927b1ca34cf46a5b02ef4 WatchSource:0}: Error finding container b33ec2b9e6c50fa9dad3f0d37c7c333bd4a8cf435f9927b1ca34cf46a5b02ef4: Status 404 returned error can't find the container with id b33ec2b9e6c50fa9dad3f0d37c7c333bd4a8cf435f9927b1ca34cf46a5b02ef4
Sep 16 18:37:07.084431 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:07.084409    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl" event={"ID":"fc2cd6fc-1c17-4997-904d-74d3b6124709","Type":"ContainerStarted","Data":"b33ec2b9e6c50fa9dad3f0d37c7c333bd4a8cf435f9927b1ca34cf46a5b02ef4"}
Sep 16 18:37:07.800989 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:07.800961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:07.906186 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:07.906117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:08.091358 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:08.091334    2562 generic.go:356] "Generic (PLEG): container finished" podID="fc2cd6fc-1c17-4997-904d-74d3b6124709" containerID="ff11590f58a17e378926294ab83580db47c06720c013f9492d3468fcdcc707a0" exitCode=0
Sep 16 18:37:08.091672 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:08.091374    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl" event={"ID":"fc2cd6fc-1c17-4997-904d-74d3b6124709","Type":"ContainerDied","Data":"ff11590f58a17e378926294ab83580db47c06720c013f9492d3468fcdcc707a0"}
Sep 16 18:37:08.551043 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:08.551015    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:08.639567 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:08.639543    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7706-3699d6"]
Sep 16 18:37:08.642824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:08.642808    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7706-3699d6"
Sep 16 18:37:08.645436 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:08.645420    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xhq6g\""
Sep 16 18:37:08.651902 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:08.651881    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7706-3699d6"]
Sep 16 18:37:08.812726 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:08.812653    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vnkt\" (UniqueName: \"kubernetes.io/projected/448c3498-c61d-4984-97cd-79c04274e794-kube-api-access-7vnkt\") pod \"test-curl-7706-3699d6\" (UID: \"448c3498-c61d-4984-97cd-79c04274e794\") " pod="opendatahub/test-curl-7706-3699d6"
Sep 16 18:37:08.914053 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:08.914026    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vnkt\" (UniqueName: \"kubernetes.io/projected/448c3498-c61d-4984-97cd-79c04274e794-kube-api-access-7vnkt\") pod \"test-curl-7706-3699d6\" (UID: \"448c3498-c61d-4984-97cd-79c04274e794\") " pod="opendatahub/test-curl-7706-3699d6"
Sep 16 18:37:08.922475 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:08.922448    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vnkt\" (UniqueName: \"kubernetes.io/projected/448c3498-c61d-4984-97cd-79c04274e794-kube-api-access-7vnkt\") pod \"test-curl-7706-3699d6\" (UID: \"448c3498-c61d-4984-97cd-79c04274e794\") " pod="opendatahub/test-curl-7706-3699d6"
Sep 16 18:37:08.952499 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:08.952478    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7706-3699d6"
Sep 16 18:37:09.225130 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:09.223938    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7706-3699d6"]
Sep 16 18:37:09.227158 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:37:09.227130    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod448c3498_c61d_4984_97cd_79c04274e794.slice/crio-bec3ee64e6c2864729d22062534751518e774f67cac6abf3c93bfb74c01739ad WatchSource:0}: Error finding container bec3ee64e6c2864729d22062534751518e774f67cac6abf3c93bfb74c01739ad: Status 404 returned error can't find the container with id bec3ee64e6c2864729d22062534751518e774f67cac6abf3c93bfb74c01739ad
Sep 16 18:37:09.229087 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:09.229068    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:37:09.240581 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:09.240560    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"
Sep 16 18:37:09.342435 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:09.342401    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:09.421563 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:09.421539    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc2cd6fc-1c17-4997-904d-74d3b6124709-maas-api-service-ca\") pod \"fc2cd6fc-1c17-4997-904d-74d3b6124709\" (UID: \"fc2cd6fc-1c17-4997-904d-74d3b6124709\") "
Sep 16 18:37:09.421857 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:09.421838    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc2cd6fc-1c17-4997-904d-74d3b6124709-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc2cd6fc-1c17-4997-904d-74d3b6124709" (UID: "fc2cd6fc-1c17-4997-904d-74d3b6124709"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:37:09.522626 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:09.522601    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc2cd6fc-1c17-4997-904d-74d3b6124709-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:37:10.103866 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:10.103840    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"
Sep 16 18:37:10.104035 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:10.103870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl" event={"ID":"fc2cd6fc-1c17-4997-904d-74d3b6124709","Type":"ContainerDied","Data":"b33ec2b9e6c50fa9dad3f0d37c7c333bd4a8cf435f9927b1ca34cf46a5b02ef4"}
Sep 16 18:37:10.104035 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:10.103938    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b33ec2b9e6c50fa9dad3f0d37c7c333bd4a8cf435f9927b1ca34cf46a5b02ef4"
Sep 16 18:37:10.105238 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:10.105215    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7706-3699d6" event={"ID":"448c3498-c61d-4984-97cd-79c04274e794","Type":"ContainerStarted","Data":"bec3ee64e6c2864729d22062534751518e774f67cac6abf3c93bfb74c01739ad"}
Sep 16 18:37:10.205789 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:10.205761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:10.954665 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:10.954630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:14.129401 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.129363    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7706-3699d6" event={"ID":"448c3498-c61d-4984-97cd-79c04274e794","Type":"ContainerStarted","Data":"0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec"}
Sep 16 18:37:14.146526 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.146482    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7706-3699d6" podStartSLOduration=2.015247797 podStartE2EDuration="6.146464627s" totalImagesPullingTime="4.13121683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:37:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:37:14.145715585 +0000 UTC m=+1254.385504329" watchObservedRunningTime="2026-09-16 18:37:14.146464627 +0000 UTC m=+1254.386253379"
Sep 16 18:37:14.788053 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.788025    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7706-3699d6"]
Sep 16 18:37:14.793861 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.793839    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7706-3699d6"]
Sep 16 18:37:14.908327 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.908272    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7706-b0a4ff"]
Sep 16 18:37:14.908691 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.908679    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="448c3498-c61d-4984-97cd-79c04274e794" containerName="test-curl-7706-3699d6"
Sep 16 18:37:14.908746 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.908694    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="448c3498-c61d-4984-97cd-79c04274e794" containerName="test-curl-7706-3699d6"
Sep 16 18:37:14.908746 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.908713    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc2cd6fc-1c17-4997-904d-74d3b6124709" containerName="revoke-keys"
Sep 16 18:37:14.908746 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.908724    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc2cd6fc-1c17-4997-904d-74d3b6124709" containerName="revoke-keys"
Sep 16 18:37:14.908839 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.908796    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="448c3498-c61d-4984-97cd-79c04274e794" containerName="test-curl-7706-3699d6"
Sep 16 18:37:14.908839 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.908808    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc2cd6fc-1c17-4997-904d-74d3b6124709" containerName="revoke-keys"
Sep 16 18:37:14.911946 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.911933    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7706-b0a4ff"
Sep 16 18:37:14.922942 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.922916    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7706-b0a4ff"]
Sep 16 18:37:14.966630 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:14.966580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmg87\" (UniqueName: \"kubernetes.io/projected/bb9c0a5f-370c-4d20-85b5-58b25466d84b-kube-api-access-tmg87\") pod \"test-curl-7706-b0a4ff\" (UID: \"bb9c0a5f-370c-4d20-85b5-58b25466d84b\") " pod="opendatahub/test-curl-7706-b0a4ff"
Sep 16 18:37:15.067495 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:15.067467    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmg87\" (UniqueName: \"kubernetes.io/projected/bb9c0a5f-370c-4d20-85b5-58b25466d84b-kube-api-access-tmg87\") pod \"test-curl-7706-b0a4ff\" (UID: \"bb9c0a5f-370c-4d20-85b5-58b25466d84b\") " pod="opendatahub/test-curl-7706-b0a4ff"
Sep 16 18:37:15.077275 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:15.077247    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmg87\" (UniqueName: \"kubernetes.io/projected/bb9c0a5f-370c-4d20-85b5-58b25466d84b-kube-api-access-tmg87\") pod \"test-curl-7706-b0a4ff\" (UID: \"bb9c0a5f-370c-4d20-85b5-58b25466d84b\") " pod="opendatahub/test-curl-7706-b0a4ff"
Sep 16 18:37:15.138923 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:15.136394    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7706-3699d6" container="test-curl-7706-3699d6"
Sep 16 18:37:15.221345 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:15.221322    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7706-b0a4ff"
Sep 16 18:37:15.499206 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:15.499181    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7706-b0a4ff"]
Sep 16 18:37:15.500883 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:37:15.500857    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb9c0a5f_370c_4d20_85b5_58b25466d84b.slice/crio-843362daa0b1c9fdfdadd35d4922abb54400b3156d4ee0105fc6585dd948947e WatchSource:0}: Error finding container 843362daa0b1c9fdfdadd35d4922abb54400b3156d4ee0105fc6585dd948947e: Status 404 returned error can't find the container with id 843362daa0b1c9fdfdadd35d4922abb54400b3156d4ee0105fc6585dd948947e
Sep 16 18:37:15.820221 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:15.820162    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:16.144120 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:16.144047    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7706-b0a4ff" event={"ID":"bb9c0a5f-370c-4d20-85b5-58b25466d84b","Type":"ContainerStarted","Data":"6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852"}
Sep 16 18:37:16.144120 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:16.144084    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7706-b0a4ff" event={"ID":"bb9c0a5f-370c-4d20-85b5-58b25466d84b","Type":"ContainerStarted","Data":"843362daa0b1c9fdfdadd35d4922abb54400b3156d4ee0105fc6585dd948947e"}
Sep 16 18:37:16.144609 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:16.144152    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7706-3699d6" podUID="448c3498-c61d-4984-97cd-79c04274e794" containerName="test-curl-7706-3699d6" containerID="cri-o://0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec" gracePeriod=2
Sep 16 18:37:16.159794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:16.159754    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7706-b0a4ff" podStartSLOduration=2.15974152 podStartE2EDuration="2.15974152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:37:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:37:16.159492058 +0000 UTC m=+1256.399280811" watchObservedRunningTime="2026-09-16 18:37:16.15974152 +0000 UTC m=+1256.399530271"
Sep 16 18:37:16.161376 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:16.161354    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7706-3699d6\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="448c3498-c61d-4984-97cd-79c04274e794" pod="opendatahub/test-curl-7706-3699d6"
Sep 16 18:37:16.163151 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:16.163132    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7706-3699d6\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="448c3498-c61d-4984-97cd-79c04274e794" pod="opendatahub/test-curl-7706-3699d6"
Sep 16 18:37:16.663219 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:16.663192    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:16.921964 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:16.921901    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7706-b0a4ff"]
Sep 16 18:37:16.925686 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:16.925655    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7706-b0a4ff"]
Sep 16 18:37:17.149733 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:17.149707    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7706-b0a4ff" container="test-curl-7706-b0a4ff"
Sep 16 18:37:17.451345 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:17.451305    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:17.565472 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:17.565433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:17.660332 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:17.660291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:17.757333 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:17.757240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:17.787403 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:17.787370    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"]
Sep 16 18:37:17.794139 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:17.794109    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ca9c2-gwfdl"]
Sep 16 18:37:17.858310 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:17.858275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:18.156209 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:18.156174    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7706-b0a4ff" podUID="bb9c0a5f-370c-4d20-85b5-58b25466d84b" containerName="test-curl-7706-b0a4ff" containerID="cri-o://6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852" gracePeriod=2
Sep 16 18:37:18.160809 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:18.160785    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7706-b0a4ff\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="bb9c0a5f-370c-4d20-85b5-58b25466d84b" pod="opendatahub/test-curl-7706-b0a4ff"
Sep 16 18:37:18.167188 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:18.167159    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:18.271584 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:18.271554    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:18.289171 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:18.289143    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc2cd6fc-1c17-4997-904d-74d3b6124709" path="/var/lib/kubelet/pods/fc2cd6fc-1c17-4997-904d-74d3b6124709/volumes"
Sep 16 18:37:18.335788 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:18.335764    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7706-3699d6"
Sep 16 18:37:18.354275 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:18.354255    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:18.397099 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:18.397063    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vnkt\" (UniqueName: \"kubernetes.io/projected/448c3498-c61d-4984-97cd-79c04274e794-kube-api-access-7vnkt\") pod \"448c3498-c61d-4984-97cd-79c04274e794\" (UID: \"448c3498-c61d-4984-97cd-79c04274e794\") "
Sep 16 18:37:18.399241 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:18.399173    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/448c3498-c61d-4984-97cd-79c04274e794-kube-api-access-7vnkt" (OuterVolumeSpecName: "kube-api-access-7vnkt") pod "448c3498-c61d-4984-97cd-79c04274e794" (UID: "448c3498-c61d-4984-97cd-79c04274e794"). InnerVolumeSpecName "kube-api-access-7vnkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:37:18.498535 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:18.498507    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vnkt\" (UniqueName: \"kubernetes.io/projected/448c3498-c61d-4984-97cd-79c04274e794-kube-api-access-7vnkt\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:37:19.163071 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:19.163030    2562 generic.go:356] "Generic (PLEG): container finished" podID="448c3498-c61d-4984-97cd-79c04274e794" containerID="0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec" exitCode=137
Sep 16 18:37:19.163071 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:19.163075    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7706-3699d6"
Sep 16 18:37:19.163573 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:19.163130    2562 scope.go:117] "RemoveContainer" containerID="0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec"
Sep 16 18:37:19.172571 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:19.172554    2562 scope.go:117] "RemoveContainer" containerID="0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec"
Sep 16 18:37:19.172858 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:19.172842    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec\": container with ID starting with 0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec not found: ID does not exist" containerID="0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec"
Sep 16 18:37:19.172918 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:19.172866    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec"} err="failed to get container status \"0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec\": rpc error: code = NotFound desc = could not find container \"0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec\": container with ID starting with 0a45f4764e0957190a4a6334f01510ab75ec58aa737832ad5557b3671f55ecec not found: ID does not exist"
Sep 16 18:37:19.173782 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:19.173760    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7706-3699d6\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="448c3498-c61d-4984-97cd-79c04274e794" pod="opendatahub/test-curl-7706-3699d6"
Sep 16 18:37:19.435110 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:19.435032    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:20.285988 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:20.285951    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7706-3699d6\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="448c3498-c61d-4984-97cd-79c04274e794" pod="opendatahub/test-curl-7706-3699d6"
Sep 16 18:37:20.287898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:20.287873    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="448c3498-c61d-4984-97cd-79c04274e794" path="/var/lib/kubelet/pods/448c3498-c61d-4984-97cd-79c04274e794/volumes"
Sep 16 18:37:20.347666 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:20.347644    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7706-b0a4ff"
Sep 16 18:37:20.349824 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:20.349804    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7706-b0a4ff\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="bb9c0a5f-370c-4d20-85b5-58b25466d84b" pod="opendatahub/test-curl-7706-b0a4ff"
Sep 16 18:37:20.366731 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:20.366710    2562 scope.go:117] "RemoveContainer" containerID="726f0004abf5eef5ad2c87dfb1f79982937dee41a80bb7a7964dcd0756deaf3d"
Sep 16 18:37:20.415122 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:20.415101    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmg87\" (UniqueName: \"kubernetes.io/projected/bb9c0a5f-370c-4d20-85b5-58b25466d84b-kube-api-access-tmg87\") pod \"bb9c0a5f-370c-4d20-85b5-58b25466d84b\" (UID: \"bb9c0a5f-370c-4d20-85b5-58b25466d84b\") "
Sep 16 18:37:20.417012 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:20.416993    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb9c0a5f-370c-4d20-85b5-58b25466d84b-kube-api-access-tmg87" (OuterVolumeSpecName: "kube-api-access-tmg87") pod "bb9c0a5f-370c-4d20-85b5-58b25466d84b" (UID: "bb9c0a5f-370c-4d20-85b5-58b25466d84b"). InnerVolumeSpecName "kube-api-access-tmg87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:37:20.515799 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:20.515779    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmg87\" (UniqueName: \"kubernetes.io/projected/bb9c0a5f-370c-4d20-85b5-58b25466d84b-kube-api-access-tmg87\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:37:20.555328 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:20.555304    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:21.176549 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:21.176520    2562 generic.go:356] "Generic (PLEG): container finished" podID="bb9c0a5f-370c-4d20-85b5-58b25466d84b" containerID="6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852" exitCode=137
Sep 16 18:37:21.176726 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:21.176560    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7706-b0a4ff"
Sep 16 18:37:21.176726 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:21.176627    2562 scope.go:117] "RemoveContainer" containerID="6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852"
Sep 16 18:37:21.179067 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:21.179038    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7706-b0a4ff\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="bb9c0a5f-370c-4d20-85b5-58b25466d84b" pod="opendatahub/test-curl-7706-b0a4ff"
Sep 16 18:37:21.185430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:21.185412    2562 scope.go:117] "RemoveContainer" containerID="6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852"
Sep 16 18:37:21.185734 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:21.185716    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852\": container with ID starting with 6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852 not found: ID does not exist" containerID="6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852"
Sep 16 18:37:21.185794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:21.185741    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852"} err="failed to get container status \"6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852\": rpc error: code = NotFound desc = could not find container \"6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852\": container with ID starting with 6a6f79773314ec6338aad3f3453ff13b326b7519e78ca54dbbbde40a694a1852 not found: ID does not exist"
Sep 16 18:37:21.187129 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:37:21.187110    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7706-b0a4ff\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="bb9c0a5f-370c-4d20-85b5-58b25466d84b" pod="opendatahub/test-curl-7706-b0a4ff"
Sep 16 18:37:21.384690 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:21.384665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:21.614672 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:21.614609    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:22.286564 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:22.286535    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb9c0a5f-370c-4d20-85b5-58b25466d84b" path="/var/lib/kubelet/pods/bb9c0a5f-370c-4d20-85b5-58b25466d84b/volumes"
Sep 16 18:37:22.501058 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:22.501025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:23.236320 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:23.236285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:23.615284 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:23.615229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:23.975104 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:23.975067    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:24.221434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:24.221402    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:26.818325 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:26.818294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:27.418299 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:27.418260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:27.587121 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:27.587087    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:28.006717 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:28.006690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:29.223166 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:29.223115    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:30.286843 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:30.286809    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:31.587381 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:31.587348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:32.572116 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:32.572072    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:32.690693 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:32.690668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:32.854834 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:32.854773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:33.522204 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:33.522166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:33.664027 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:33.664001    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:50.809210 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:50.809178    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:51.118067 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:51.117987    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:51.231469 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:51.231437    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:51.292612 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:51.292575    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:51.693659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:51.693626    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:52.044642 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:52.044540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:52.230191 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:52.230165    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:54.998510 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:54.998470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:55.568746 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:55.568702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:55.675536 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:55.675504    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:56.083781 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:56.083749    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:57.130950 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:57.130921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:57.508079 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:57.508050    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:57.928755 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:57.928729    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:58.186679 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:58.186612    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:58.786928 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:58.786897    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:58.887831 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:58.887800    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:59.727998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:37:59.727945    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:00.744616 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:00.744559    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:00.923236 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:00.923202    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:01.396896 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.396849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:01.498946 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.498916    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:01.572182 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.572158    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"]
Sep 16 18:38:01.572545 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.572534    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb9c0a5f-370c-4d20-85b5-58b25466d84b" containerName="test-curl-7706-b0a4ff"
Sep 16 18:38:01.572614 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.572548    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb9c0a5f-370c-4d20-85b5-58b25466d84b" containerName="test-curl-7706-b0a4ff"
Sep 16 18:38:01.572664 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.572646    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb9c0a5f-370c-4d20-85b5-58b25466d84b" containerName="test-curl-7706-b0a4ff"
Sep 16 18:38:01.575827 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.575813    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:38:01.578424 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.578404    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-972df0-dockercfg-z9zgn\""
Sep 16 18:38:01.578535 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.578452    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-972df0\""
Sep 16 18:38:01.587183 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.587162    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"]
Sep 16 18:38:01.669811 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.669792    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"]
Sep 16 18:38:01.673161 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.673148    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:38:01.685330 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.685310    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"]
Sep 16 18:38:01.752341 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.752319    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7nd4\" (UniqueName: \"kubernetes.io/projected/c66061a2-cad0-4f3a-9c6a-83123e79badf-kube-api-access-s7nd4\") pod \"payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt\" (UID: \"c66061a2-cad0-4f3a-9c6a-83123e79badf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:38:01.752667 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.752381    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c66061a2-cad0-4f3a-9c6a-83123e79badf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt\" (UID: \"c66061a2-cad0-4f3a-9c6a-83123e79badf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:38:01.853809 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.853786    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34e3086f-ddc0-4550-94e4-da0e04a8c127-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6\" (UID: \"34e3086f-ddc0-4550-94e4-da0e04a8c127\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:38:01.853919 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.853900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7nd4\" (UniqueName: \"kubernetes.io/projected/c66061a2-cad0-4f3a-9c6a-83123e79badf-kube-api-access-s7nd4\") pod \"payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt\" (UID: \"c66061a2-cad0-4f3a-9c6a-83123e79badf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:38:01.853999 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.853982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c66061a2-cad0-4f3a-9c6a-83123e79badf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt\" (UID: \"c66061a2-cad0-4f3a-9c6a-83123e79badf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:38:01.854060 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.854026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp9wb\" (UniqueName: \"kubernetes.io/projected/34e3086f-ddc0-4550-94e4-da0e04a8c127-kube-api-access-xp9wb\") pod \"payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6\" (UID: \"34e3086f-ddc0-4550-94e4-da0e04a8c127\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:38:01.854712 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.854693    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c66061a2-cad0-4f3a-9c6a-83123e79badf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt\" (UID: \"c66061a2-cad0-4f3a-9c6a-83123e79badf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:38:01.864103 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.864080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7nd4\" (UniqueName: \"kubernetes.io/projected/c66061a2-cad0-4f3a-9c6a-83123e79badf-kube-api-access-s7nd4\") pod \"payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt\" (UID: \"c66061a2-cad0-4f3a-9c6a-83123e79badf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:38:01.886651 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.886633    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:38:01.955281 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.955253    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp9wb\" (UniqueName: \"kubernetes.io/projected/34e3086f-ddc0-4550-94e4-da0e04a8c127-kube-api-access-xp9wb\") pod \"payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6\" (UID: \"34e3086f-ddc0-4550-94e4-da0e04a8c127\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:38:01.955409 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.955298    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34e3086f-ddc0-4550-94e4-da0e04a8c127-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6\" (UID: \"34e3086f-ddc0-4550-94e4-da0e04a8c127\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:38:01.956003 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.955985    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34e3086f-ddc0-4550-94e4-da0e04a8c127-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6\" (UID: \"34e3086f-ddc0-4550-94e4-da0e04a8c127\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:38:01.966489 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.966469    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp9wb\" (UniqueName: \"kubernetes.io/projected/34e3086f-ddc0-4550-94e4-da0e04a8c127-kube-api-access-xp9wb\") pod \"payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6\" (UID: \"34e3086f-ddc0-4550-94e4-da0e04a8c127\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:38:01.982494 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:01.982473    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:38:02.062927 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.062896    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:02.170373 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.170351    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"]
Sep 16 18:38:02.171751 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:38:02.171732    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc66061a2_cad0_4f3a_9c6a_83123e79badf.slice/crio-1da1104f8cf0a601bf3220bb192549fe53acf73ef5fe5f6ec6f06f71570283f4 WatchSource:0}: Error finding container 1da1104f8cf0a601bf3220bb192549fe53acf73ef5fe5f6ec6f06f71570283f4: Status 404 returned error can't find the container with id 1da1104f8cf0a601bf3220bb192549fe53acf73ef5fe5f6ec6f06f71570283f4
Sep 16 18:38:02.256806 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.256781    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"]
Sep 16 18:38:02.258309 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:38:02.258285    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34e3086f_ddc0_4550_94e4_da0e04a8c127.slice/crio-a35fb3ff7f6024d8db965549622a011b20ed7eb36f8b4929897ef9428e9852cb WatchSource:0}: Error finding container a35fb3ff7f6024d8db965549622a011b20ed7eb36f8b4929897ef9428e9852cb: Status 404 returned error can't find the container with id a35fb3ff7f6024d8db965549622a011b20ed7eb36f8b4929897ef9428e9852cb
Sep 16 18:38:02.391315 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.391283    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt" event={"ID":"c66061a2-cad0-4f3a-9c6a-83123e79badf","Type":"ContainerStarted","Data":"3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9"}
Sep 16 18:38:02.391484 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.391340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt" event={"ID":"c66061a2-cad0-4f3a-9c6a-83123e79badf","Type":"ContainerStarted","Data":"1da1104f8cf0a601bf3220bb192549fe53acf73ef5fe5f6ec6f06f71570283f4"}
Sep 16 18:38:02.391484 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.391434    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:38:02.392936 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.392912    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6" event={"ID":"34e3086f-ddc0-4550-94e4-da0e04a8c127","Type":"ContainerStarted","Data":"9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2"}
Sep 16 18:38:02.393030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.392951    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6" event={"ID":"34e3086f-ddc0-4550-94e4-da0e04a8c127","Type":"ContainerStarted","Data":"a35fb3ff7f6024d8db965549622a011b20ed7eb36f8b4929897ef9428e9852cb"}
Sep 16 18:38:02.393030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.392987    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:38:02.412312 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.412269    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt" podStartSLOduration=1.4122569280000001 podStartE2EDuration="1.412256928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:38:02.409716503 +0000 UTC m=+1302.649505267" watchObservedRunningTime="2026-09-16 18:38:02.412256928 +0000 UTC m=+1302.652045679"
Sep 16 18:38:02.432975 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.432925    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6" podStartSLOduration=1.432908216 podStartE2EDuration="1.432908216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:38:02.429389715 +0000 UTC m=+1302.669178469" watchObservedRunningTime="2026-09-16 18:38:02.432908216 +0000 UTC m=+1302.672696971"
Sep 16 18:38:02.698333 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.698308    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"]
Sep 16 18:38:02.711465 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:02.711427    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:03.044635 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:03.044540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:03.991305 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:03.991277    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:13.400636 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:13.400572    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:38:13.401048 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:13.400786    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:38:17.718261 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.718213    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg" podUID="edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" containerName="main" containerID="cri-o://43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537" gracePeriod=45
Sep 16 18:38:17.975980 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.975914    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:38:17.992064 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992040    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-home\") pod \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") "
Sep 16 18:38:17.992171 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992080    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfgsr\" (UniqueName: \"kubernetes.io/projected/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-kube-api-access-hfgsr\") pod \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") "
Sep 16 18:38:17.992171 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992099    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tls-certs\") pod \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") "
Sep 16 18:38:17.992171 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992125    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tmp-dir\") pod \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") "
Sep 16 18:38:17.992171 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992153    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-dshm\") pod \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") "
Sep 16 18:38:17.992378 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992295    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-model-cache\") pod \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\" (UID: \"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6\") "
Sep 16 18:38:17.992378 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992307    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-home" (OuterVolumeSpecName: "home") pod "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" (UID: "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:38:17.992486 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992440    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" (UID: "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:38:17.992641 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992611    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-model-cache" (OuterVolumeSpecName: "model-cache") pod "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" (UID: "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:38:17.992899 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992732    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:17.992899 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992749    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:17.992899 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.992763    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:17.994411 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.994387    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" (UID: "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:38:17.994411 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.994398    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-dshm" (OuterVolumeSpecName: "dshm") pod "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" (UID: "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:38:17.994535 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:17.994461    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-kube-api-access-hfgsr" (OuterVolumeSpecName: "kube-api-access-hfgsr") pod "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" (UID: "edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6"). InnerVolumeSpecName "kube-api-access-hfgsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:38:18.093500 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.093477    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfgsr\" (UniqueName: \"kubernetes.io/projected/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-kube-api-access-hfgsr\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:18.093632 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.093501    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:18.093632 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.093515    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:18.476540 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.476508    2562 generic.go:356] "Generic (PLEG): container finished" podID="edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" containerID="43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537" exitCode=0
Sep 16 18:38:18.476671 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.476559    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"
Sep 16 18:38:18.476671 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.476618    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg" event={"ID":"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6","Type":"ContainerDied","Data":"43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537"}
Sep 16 18:38:18.476671 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.476660    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg" event={"ID":"edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6","Type":"ContainerDied","Data":"de35371cbe175b9f767028ee32b70e45af240a28ebfa2d356fad6358963f4e50"}
Sep 16 18:38:18.476773 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.476680    2562 scope.go:117] "RemoveContainer" containerID="43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537"
Sep 16 18:38:18.485523 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.485507    2562 scope.go:117] "RemoveContainer" containerID="43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537"
Sep 16 18:38:18.485798 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:38:18.485780    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537\": container with ID starting with 43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537 not found: ID does not exist" containerID="43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537"
Sep 16 18:38:18.485844 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.485805    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537"} err="failed to get container status \"43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537\": rpc error: code = NotFound desc = could not find container \"43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537\": container with ID starting with 43681206c665f146f7973d725899c8aa64722f5e66917c4c7de4c4f2d79ff537 not found: ID does not exist"
Sep 16 18:38:18.498045 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.498024    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"]
Sep 16 18:38:18.501533 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.501514    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-b-15e17679-kserve-789cd97b8d-rpdvg"]
Sep 16 18:38:18.932355 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:18.932323    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:20.286547 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:20.286518    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" path="/var/lib/kubelet/pods/edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6/volumes"
Sep 16 18:38:20.399747 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:20.399716    2562 scope.go:117] "RemoveContainer" containerID="c0ad52bdaf365fd633a5f225ffdad5f1d5a3f15ca9084bccf9a558cfea4b38a3"
Sep 16 18:38:23.564197 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:23.564170    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:24.628401 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:24.628338    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:25.521691 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:25.521658    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:25.633335 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:25.633311    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:25.958115 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:25.958079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:26.335686 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:26.335612    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:26.542379 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:26.542349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:26.942924 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:26.942890    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:27.605883 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:27.605851    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:28.557066 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:28.557023    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:29.691646 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:29.691617    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:29.925369 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:29.925331    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:30.037369 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:30.037297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:31.191264 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:31.191202    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:32.394464 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:32.394414    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:33.321908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:33.321876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:34.111228 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:34.111197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:34.920855 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:34.920809    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:35.643104 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:35.643079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:36.524442 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:36.524409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:36.628734 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:36.628702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:36.854009 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:36.853932    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:37.249753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.249728    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"]
Sep 16 18:38:37.250233 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.250216    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" containerName="main"
Sep 16 18:38:37.250233 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.250237    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" containerName="main"
Sep 16 18:38:37.250412 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.250340    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="edc06a7c-78f7-4ecf-9293-90d4ce5cbdf6" containerName="main"
Sep 16 18:38:37.254982 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.254954    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:38:37.257268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.257245    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-d3e8be-dockercfg-r45td\""
Sep 16 18:38:37.257395 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.257297    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-d3e8be\""
Sep 16 18:38:37.257395 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.257238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:37.257577 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.257561    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"]
Sep 16 18:38:37.261008 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.260994    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:38:37.265249 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.265229    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"]
Sep 16 18:38:37.273133 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.273112    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"]
Sep 16 18:38:37.351816 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.351789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:37.356905 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.356887    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx7wk\" (UniqueName: \"kubernetes.io/projected/66ae44f1-788c-4729-a024-e1e0fcf35c05-kube-api-access-hx7wk\") pod \"payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt\" (UID: \"66ae44f1-788c-4729-a024-e1e0fcf35c05\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:38:37.356981 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.356917    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ae44f1-788c-4729-a024-e1e0fcf35c05-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt\" (UID: \"66ae44f1-788c-4729-a024-e1e0fcf35c05\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:38:37.356981 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.356961    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbfln\" (UniqueName: \"kubernetes.io/projected/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-kube-api-access-mbfln\") pod \"payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5\" (UID: \"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:38:37.357054 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.356983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5\" (UID: \"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:38:37.457816 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.457791    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx7wk\" (UniqueName: \"kubernetes.io/projected/66ae44f1-788c-4729-a024-e1e0fcf35c05-kube-api-access-hx7wk\") pod \"payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt\" (UID: \"66ae44f1-788c-4729-a024-e1e0fcf35c05\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:38:37.457912 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.457822    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ae44f1-788c-4729-a024-e1e0fcf35c05-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt\" (UID: \"66ae44f1-788c-4729-a024-e1e0fcf35c05\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:38:37.457912 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.457857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbfln\" (UniqueName: \"kubernetes.io/projected/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-kube-api-access-mbfln\") pod \"payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5\" (UID: \"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:38:37.457912 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.457906    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5\" (UID: \"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:38:37.458476 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.458460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ae44f1-788c-4729-a024-e1e0fcf35c05-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt\" (UID: \"66ae44f1-788c-4729-a024-e1e0fcf35c05\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:38:37.458621 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.458607    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5\" (UID: \"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:38:37.466409 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.466388    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx7wk\" (UniqueName: \"kubernetes.io/projected/66ae44f1-788c-4729-a024-e1e0fcf35c05-kube-api-access-hx7wk\") pod \"payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt\" (UID: \"66ae44f1-788c-4729-a024-e1e0fcf35c05\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:38:37.466505 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.466485    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbfln\" (UniqueName: \"kubernetes.io/projected/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-kube-api-access-mbfln\") pod \"payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5\" (UID: \"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:38:37.566138 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.566089    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:38:37.576217 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.576202    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:38:37.853547 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.853522    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"]
Sep 16 18:38:37.855868 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:38:37.855843    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee3a65c_4999_4bfe_bcee_f81cfb502cdc.slice/crio-a95a28cadf17bb0b97a4b7a4b65633249361b417d49476c560e68be91077f0a5 WatchSource:0}: Error finding container a95a28cadf17bb0b97a4b7a4b65633249361b417d49476c560e68be91077f0a5: Status 404 returned error can't find the container with id a95a28cadf17bb0b97a4b7a4b65633249361b417d49476c560e68be91077f0a5
Sep 16 18:38:37.882068 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:37.882047    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"]
Sep 16 18:38:37.887633 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:38:37.887611    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66ae44f1_788c_4729_a024_e1e0fcf35c05.slice/crio-e074fcf7dc9b79d110a424c1c8b3bd9c366771f727f8d989a0a795a37388492d WatchSource:0}: Error finding container e074fcf7dc9b79d110a424c1c8b3bd9c366771f727f8d989a0a795a37388492d: Status 404 returned error can't find the container with id e074fcf7dc9b79d110a424c1c8b3bd9c366771f727f8d989a0a795a37388492d
Sep 16 18:38:38.595193 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:38.595160    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt" event={"ID":"66ae44f1-788c-4729-a024-e1e0fcf35c05","Type":"ContainerStarted","Data":"47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8"}
Sep 16 18:38:38.595355 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:38.595220    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:38:38.595355 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:38.595244    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt" event={"ID":"66ae44f1-788c-4729-a024-e1e0fcf35c05","Type":"ContainerStarted","Data":"e074fcf7dc9b79d110a424c1c8b3bd9c366771f727f8d989a0a795a37388492d"}
Sep 16 18:38:38.596751 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:38.596724    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5" event={"ID":"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc","Type":"ContainerStarted","Data":"2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7"}
Sep 16 18:38:38.596869 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:38.596775    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:38:38.596869 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:38.596795    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5" event={"ID":"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc","Type":"ContainerStarted","Data":"a95a28cadf17bb0b97a4b7a4b65633249361b417d49476c560e68be91077f0a5"}
Sep 16 18:38:38.614197 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:38.614157    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt" podStartSLOduration=1.614144627 podStartE2EDuration="1.614144627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:38:38.612324712 +0000 UTC m=+1338.852113463" watchObservedRunningTime="2026-09-16 18:38:38.614144627 +0000 UTC m=+1338.853933378"
Sep 16 18:38:38.631815 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:38.631768    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5" podStartSLOduration=1.631750254 podStartE2EDuration="1.631750254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:38:38.628889546 +0000 UTC m=+1338.868678298" watchObservedRunningTime="2026-09-16 18:38:38.631750254 +0000 UTC m=+1338.871539011"
Sep 16 18:38:41.586340 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:41.586304    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"]
Sep 16 18:38:41.608302 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:41.608275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:48.875670 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:48.875637    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:49.605622 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:49.605580    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:38:49.605803 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:49.605771    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:38:56.605071 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:56.604975    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns" podUID="96c07cc6-17d9-40ce-9c4a-ea90783a6363" containerName="main" containerID="cri-o://3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725" gracePeriod=45
Sep 16 18:38:56.862492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:56.862440    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:38:57.018666 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.018640    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-model-cache\") pod \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") "
Sep 16 18:38:57.018796 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.018685    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tls-certs\") pod \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") "
Sep 16 18:38:57.018796 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.018729    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-dshm\") pod \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") "
Sep 16 18:38:57.018796 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.018767    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl8br\" (UniqueName: \"kubernetes.io/projected/96c07cc6-17d9-40ce-9c4a-ea90783a6363-kube-api-access-hl8br\") pod \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") "
Sep 16 18:38:57.018964 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.018817    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tmp-dir\") pod \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") "
Sep 16 18:38:57.018964 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.018861    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-home\") pod \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\" (UID: \"96c07cc6-17d9-40ce-9c4a-ea90783a6363\") "
Sep 16 18:38:57.018964 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.018901    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-model-cache" (OuterVolumeSpecName: "model-cache") pod "96c07cc6-17d9-40ce-9c4a-ea90783a6363" (UID: "96c07cc6-17d9-40ce-9c4a-ea90783a6363"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:38:57.019154 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.019104    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "96c07cc6-17d9-40ce-9c4a-ea90783a6363" (UID: "96c07cc6-17d9-40ce-9c4a-ea90783a6363"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:38:57.019220 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.019153    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-home" (OuterVolumeSpecName: "home") pod "96c07cc6-17d9-40ce-9c4a-ea90783a6363" (UID: "96c07cc6-17d9-40ce-9c4a-ea90783a6363"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:38:57.019220 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.019191    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:57.020860 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.020837    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "96c07cc6-17d9-40ce-9c4a-ea90783a6363" (UID: "96c07cc6-17d9-40ce-9c4a-ea90783a6363"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:38:57.020992 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.020974    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c07cc6-17d9-40ce-9c4a-ea90783a6363-kube-api-access-hl8br" (OuterVolumeSpecName: "kube-api-access-hl8br") pod "96c07cc6-17d9-40ce-9c4a-ea90783a6363" (UID: "96c07cc6-17d9-40ce-9c4a-ea90783a6363"). InnerVolumeSpecName "kube-api-access-hl8br". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:38:57.021042 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.021010    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-dshm" (OuterVolumeSpecName: "dshm") pod "96c07cc6-17d9-40ce-9c4a-ea90783a6363" (UID: "96c07cc6-17d9-40ce-9c4a-ea90783a6363"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:38:57.119990 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.119930    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:57.119990 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.119952    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl8br\" (UniqueName: \"kubernetes.io/projected/96c07cc6-17d9-40ce-9c4a-ea90783a6363-kube-api-access-hl8br\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:57.119990 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.119966    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:57.119990 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.119977    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96c07cc6-17d9-40ce-9c4a-ea90783a6363-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:57.119990 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.119988    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96c07cc6-17d9-40ce-9c4a-ea90783a6363-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:38:57.697316 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.697289    2562 generic.go:356] "Generic (PLEG): container finished" podID="96c07cc6-17d9-40ce-9c4a-ea90783a6363" containerID="3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725" exitCode=0
Sep 16 18:38:57.697699 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.697334    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"
Sep 16 18:38:57.697699 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.697368    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns" event={"ID":"96c07cc6-17d9-40ce-9c4a-ea90783a6363","Type":"ContainerDied","Data":"3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725"}
Sep 16 18:38:57.697699 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.697408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns" event={"ID":"96c07cc6-17d9-40ce-9c4a-ea90783a6363","Type":"ContainerDied","Data":"c3800d5a66c43f72c35fadcc27ab6b2ff7f155fb5d171b23b0482e7aff6155e2"}
Sep 16 18:38:57.697699 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.697424    2562 scope.go:117] "RemoveContainer" containerID="3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725"
Sep 16 18:38:57.707192 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.707176    2562 scope.go:117] "RemoveContainer" containerID="3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725"
Sep 16 18:38:57.707466 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:38:57.707443    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725\": container with ID starting with 3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725 not found: ID does not exist" containerID="3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725"
Sep 16 18:38:57.707516 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.707474    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725"} err="failed to get container status \"3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725\": rpc error: code = NotFound desc = could not find container \"3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725\": container with ID starting with 3a1495c8f3f4a19aae655826541139dd127361209722143a068ef889964d9725 not found: ID does not exist"
Sep 16 18:38:57.720272 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.720249    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"]
Sep 16 18:38:57.725205 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:57.725186    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/e2e-conflict-a-15e17679-kserve-7c57f675b6-krbns"]
Sep 16 18:38:58.285911 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:58.285883    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96c07cc6-17d9-40ce-9c4a-ea90783a6363" path="/var/lib/kubelet/pods/96c07cc6-17d9-40ce-9c4a-ea90783a6363/volumes"
Sep 16 18:38:59.882260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:38:59.882232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:00.000707 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:00.000683    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:00.929285 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:00.929248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:01.805006 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:01.804974    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:02.641427 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:02.641402    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:03.417031 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:03.417006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:04.233470 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:04.233435    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:04.977260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:04.977226    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:05.846559 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:05.846520    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:20.426423 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:20.426393    2562 scope.go:117] "RemoveContainer" containerID="396cd57bb495dba4ff3424bac23459d649fec724fddb076fc889033726b3722a"
Sep 16 18:39:20.435350 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:20.435333    2562 scope.go:117] "RemoveContainer" containerID="3f1bbf1f90513a22c2f9c7cb80b4d76db74f397338bbf77369fc95bc5c1d67af"
Sep 16 18:39:20.443287 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:20.443267    2562 scope.go:117] "RemoveContainer" containerID="5f84eaff4611a940f66347ae604fac7a06a59bbf074f3572f85535e9a5197aa1"
Sep 16 18:39:28.826552 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:28.826521    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"]
Sep 16 18:39:28.827130 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:28.827111    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c07cc6-17d9-40ce-9c4a-ea90783a6363" containerName="main"
Sep 16 18:39:28.827226 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:28.827132    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c07cc6-17d9-40ce-9c4a-ea90783a6363" containerName="main"
Sep 16 18:39:28.827280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:28.827249    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c07cc6-17d9-40ce-9c4a-ea90783a6363" containerName="main"
Sep 16 18:39:28.831757 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:28.831739    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"
Sep 16 18:39:28.834449 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:28.834432    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:39:28.834561 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:28.834543    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:39:28.844334 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:28.844313    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"]
Sep 16 18:39:28.878608 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:28.878563    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416a4bbd-c512-4eac-a86b-96d6eb6ac5e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a462d953-7vxwh\" (UID: \"416a4bbd-c512-4eac-a86b-96d6eb6ac5e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"
Sep 16 18:39:28.979494 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:28.979476    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416a4bbd-c512-4eac-a86b-96d6eb6ac5e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a462d953-7vxwh\" (UID: \"416a4bbd-c512-4eac-a86b-96d6eb6ac5e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"
Sep 16 18:39:28.980050 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:28.980033    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416a4bbd-c512-4eac-a86b-96d6eb6ac5e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a462d953-7vxwh\" (UID: \"416a4bbd-c512-4eac-a86b-96d6eb6ac5e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"
Sep 16 18:39:29.035449 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:29.035425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:29.141887 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:29.141840    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"
Sep 16 18:39:29.417754 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:29.417727    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"]
Sep 16 18:39:29.419294 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:39:29.419267    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416a4bbd_c512_4eac_a86b_96d6eb6ac5e8.slice/crio-86215e269d21d824dd85eb7dedfda051bd577d7c50a395fbfce08093ed855924 WatchSource:0}: Error finding container 86215e269d21d824dd85eb7dedfda051bd577d7c50a395fbfce08093ed855924: Status 404 returned error can't find the container with id 86215e269d21d824dd85eb7dedfda051bd577d7c50a395fbfce08093ed855924
Sep 16 18:39:29.871296 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:29.871270    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh" event={"ID":"416a4bbd-c512-4eac-a86b-96d6eb6ac5e8","Type":"ContainerStarted","Data":"7ca15c7a022e07400b1ece3016b3869d0ae8f51362d3aa264b7d82dda21a819c"}
Sep 16 18:39:29.871603 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:29.871316    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh" event={"ID":"416a4bbd-c512-4eac-a86b-96d6eb6ac5e8","Type":"ContainerStarted","Data":"86215e269d21d824dd85eb7dedfda051bd577d7c50a395fbfce08093ed855924"}
Sep 16 18:39:29.894941 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:29.894891    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh" podStartSLOduration=1.8948779789999999 podStartE2EDuration="1.894877979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:39:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:29.89108125 +0000 UTC m=+1390.130870001" watchObservedRunningTime="2026-09-16 18:39:29.894877979 +0000 UTC m=+1390.134666732"
Sep 16 18:39:30.154831 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:30.154808    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:30.877989 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:30.877965    2562 generic.go:356] "Generic (PLEG): container finished" podID="416a4bbd-c512-4eac-a86b-96d6eb6ac5e8" containerID="7ca15c7a022e07400b1ece3016b3869d0ae8f51362d3aa264b7d82dda21a819c" exitCode=0
Sep 16 18:39:30.878325 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:30.878052    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh" event={"ID":"416a4bbd-c512-4eac-a86b-96d6eb6ac5e8","Type":"ContainerDied","Data":"7ca15c7a022e07400b1ece3016b3869d0ae8f51362d3aa264b7d82dda21a819c"}
Sep 16 18:39:31.326536 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:31.326505    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:32.012719 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:32.012695    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"
Sep 16 18:39:32.106825 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:32.106799    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416a4bbd-c512-4eac-a86b-96d6eb6ac5e8-maas-api-service-ca\") pod \"416a4bbd-c512-4eac-a86b-96d6eb6ac5e8\" (UID: \"416a4bbd-c512-4eac-a86b-96d6eb6ac5e8\") "
Sep 16 18:39:32.107138 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:32.107120    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/416a4bbd-c512-4eac-a86b-96d6eb6ac5e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "416a4bbd-c512-4eac-a86b-96d6eb6ac5e8" (UID: "416a4bbd-c512-4eac-a86b-96d6eb6ac5e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:39:32.207689 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:32.207665    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416a4bbd-c512-4eac-a86b-96d6eb6ac5e8-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:39:32.213301 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:32.213277    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:32.889346 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:32.889323    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"
Sep 16 18:39:32.889468 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:32.889358    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh" event={"ID":"416a4bbd-c512-4eac-a86b-96d6eb6ac5e8","Type":"ContainerDied","Data":"86215e269d21d824dd85eb7dedfda051bd577d7c50a395fbfce08093ed855924"}
Sep 16 18:39:32.889468 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:32.889398    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86215e269d21d824dd85eb7dedfda051bd577d7c50a395fbfce08093ed855924"
Sep 16 18:39:33.099299 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:33.099274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:34.021293 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:34.021247    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:34.728868 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:34.728840    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:35.586156 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:35.586126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:36.480212 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:36.480182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:36.937536 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:36.937511    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"]
Sep 16 18:39:36.937941 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:36.937928    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="416a4bbd-c512-4eac-a86b-96d6eb6ac5e8" containerName="revoke-keys"
Sep 16 18:39:36.937989 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:36.937942    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="416a4bbd-c512-4eac-a86b-96d6eb6ac5e8" containerName="revoke-keys"
Sep 16 18:39:36.938047 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:36.938039    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="416a4bbd-c512-4eac-a86b-96d6eb6ac5e8" containerName="revoke-keys"
Sep 16 18:39:36.941154 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:36.941140    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:36.945673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:36.945654    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-6942a4\"/\"distinct-w4-6942a4-kserve-self-signed-certs\""
Sep 16 18:39:36.957275 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:36.957248    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"]
Sep 16 18:39:36.988177 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:36.988153    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:37.054297 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.054275    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-home\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.054394 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.054318    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-tmp-dir\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.054435 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.054393    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-model-cache\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.054435 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.054419    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e718cdac-500a-478f-8e6d-3951cdd636b0-tls-certs\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.054504 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.054441    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-dshm\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.054504 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.054465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqjhb\" (UniqueName: \"kubernetes.io/projected/e718cdac-500a-478f-8e6d-3951cdd636b0-kube-api-access-gqjhb\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.155173 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.155151    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqjhb\" (UniqueName: \"kubernetes.io/projected/e718cdac-500a-478f-8e6d-3951cdd636b0-kube-api-access-gqjhb\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.155278 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.155187    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-home\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.155278 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.155225    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-tmp-dir\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.155384 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.155276    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-model-cache\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.155384 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.155301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e718cdac-500a-478f-8e6d-3951cdd636b0-tls-certs\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.155384 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.155326    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-dshm\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.155632 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.155578    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-home\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.155632 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.155619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-model-cache\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.155765 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.155654    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-tmp-dir\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.157669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.157651    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-dshm\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.157899 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.157883    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e718cdac-500a-478f-8e6d-3951cdd636b0-tls-certs\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.164449 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.164426    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqjhb\" (UniqueName: \"kubernetes.io/projected/e718cdac-500a-478f-8e6d-3951cdd636b0-kube-api-access-gqjhb\") pod \"distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") " pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.250811 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.250759    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:37.531849 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.531825    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"]
Sep 16 18:39:37.533857 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:39:37.533827    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode718cdac_500a_478f_8e6d_3951cdd636b0.slice/crio-10520f6c83333d769d91f3576c24e174a87700dc34aae5aaa1aabbb268accfbc WatchSource:0}: Error finding container 10520f6c83333d769d91f3576c24e174a87700dc34aae5aaa1aabbb268accfbc: Status 404 returned error can't find the container with id 10520f6c83333d769d91f3576c24e174a87700dc34aae5aaa1aabbb268accfbc
Sep 16 18:39:37.918872 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.918839    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k" event={"ID":"e718cdac-500a-478f-8e6d-3951cdd636b0","Type":"ContainerStarted","Data":"1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46"}
Sep 16 18:39:37.919037 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.918879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k" event={"ID":"e718cdac-500a-478f-8e6d-3951cdd636b0","Type":"ContainerStarted","Data":"10520f6c83333d769d91f3576c24e174a87700dc34aae5aaa1aabbb268accfbc"}
Sep 16 18:39:37.939181 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:37.939139    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k" podStartSLOduration=1.93912313 podStartE2EDuration="1.93912313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:39:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:37.937994301 +0000 UTC m=+1398.177783054" watchObservedRunningTime="2026-09-16 18:39:37.93912313 +0000 UTC m=+1398.178911882"
Sep 16 18:39:44.536928 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:44.536880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:44.632826 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:44.632777    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:44.733430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:44.733400    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:44.831435 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:44.831354    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"]
Sep 16 18:39:44.833823 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:44.833800    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a462d953-7vxwh"]
Sep 16 18:39:44.861122 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:44.861097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:45.327555 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:45.327528    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:46.202724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:46.202698    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:46.285647 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:46.285621    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="416a4bbd-c512-4eac-a86b-96d6eb6ac5e8" path="/var/lib/kubelet/pods/416a4bbd-c512-4eac-a86b-96d6eb6ac5e8/volumes"
Sep 16 18:39:47.251131 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:47.251098    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:47.251557 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:47.251160    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:47.260160 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:47.260123    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:47.263398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:47.263378    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:47.877854 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:47.877830    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:47.984290 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:47.984265    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:39:48.014646 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:48.014571    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:48.741697 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:48.741666    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:49.620797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:49.620757    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:50.443855 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:50.443820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:39:52.940088 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:39:52.940053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:08.720266 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:40:08.720235    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:16.657223 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:40:16.657191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:30.089956 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:40:30.089925    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:31.081560 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:40:31.081527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:37.205732 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:40:37.205699    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:03.110760 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:03.110723    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:06.188677 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:06.188637    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:18.279919 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:18.279885    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:19.685166 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:19.685134    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:19.861073 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:19.861038    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:19.999061 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:19.999001    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:20.487708 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:20.487684    2562 scope.go:117] "RemoveContainer" containerID="e4efdba3fd57935981dc5f67e321a7636fdd2cbe295bd58276323efaac40962f"
Sep 16 18:41:20.750702 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:20.750641    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:20.905559 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:20.905532    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:22.442113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:22.442079    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:24.429124 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:24.429087    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:27.099924 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:27.099891    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:27.256113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:27.256086    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:32.382469 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:32.382434    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:32.692377 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:32.692342    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:32.904817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:32.904783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:33.574476 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:33.574452    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:43.121793 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:43.121708    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:43.244173 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:43.244144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:44.290129 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:44.290093    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:53.314554 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:53.314521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:53.575029 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:53.574954    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:53.867797 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:41:53.867733    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:04.465456 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:04.465420    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:14.562269 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:14.562232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:16.710329 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:16.710280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:20.507656 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:20.507628    2562 scope.go:117] "RemoveContainer" containerID="2b315c3ef3b3e3f4d202f485b0160968650ecffb393246628c538a54f83d7074"
Sep 16 18:42:21.813803 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:21.813768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:21.901808 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:21.901781    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:22.237087 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:22.237059    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:22.774613 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:22.774553    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:22.860351 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:22.860325    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:25.414218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:25.414178    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:26.306303 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:26.306270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:26.404217 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:26.404173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:27.247612 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:27.247557    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:28.249073 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:28.249040    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:29.138934 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:29.138905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:30.007813 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:30.007773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:30.741118 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:30.741080    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:31.494127 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:31.494085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:31.593560 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:31.593532    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:31.990069 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:31.990041    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:32.170810 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.170768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:32.274099 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.274021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:32.277289 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.277267    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"]
Sep 16 18:42:32.282309 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.282293    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:42:32.284980 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.284957    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-6a7741\""
Sep 16 18:42:32.285078 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.285010    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-6a7741-dockercfg-7p9gr\""
Sep 16 18:42:32.294311 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.294292    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"]
Sep 16 18:42:32.404317 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.404288    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt\" (UID: \"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:42:32.404432 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.404362    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqr7q\" (UniqueName: \"kubernetes.io/projected/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-kube-api-access-vqr7q\") pod \"payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt\" (UID: \"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:42:32.505124 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.505098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt\" (UID: \"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:42:32.505476 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.505137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqr7q\" (UniqueName: \"kubernetes.io/projected/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-kube-api-access-vqr7q\") pod \"payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt\" (UID: \"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:42:32.505741 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.505724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt\" (UID: \"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:42:32.516524 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.516505    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqr7q\" (UniqueName: \"kubernetes.io/projected/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-kube-api-access-vqr7q\") pod \"payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt\" (UID: \"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f\") " pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:42:32.594259 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.594210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:42:32.867334 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:32.867251    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"]
Sep 16 18:42:32.879431 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:42:32.879406    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod941ab0a7_e2b9_4a22_888a_69bc0e19cb6f.slice/crio-d9c409583e8a78eb919c25ed5689a72b353740eb962fe809ffac6e873af3f757 WatchSource:0}: Error finding container d9c409583e8a78eb919c25ed5689a72b353740eb962fe809ffac6e873af3f757: Status 404 returned error can't find the container with id d9c409583e8a78eb919c25ed5689a72b353740eb962fe809ffac6e873af3f757
Sep 16 18:42:33.773061 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:33.773031    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:33.827965 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:33.827936    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt" event={"ID":"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f","Type":"ContainerStarted","Data":"0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103"}
Sep 16 18:42:33.828124 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:33.827981    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt" event={"ID":"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f","Type":"ContainerStarted","Data":"d9c409583e8a78eb919c25ed5689a72b353740eb962fe809ffac6e873af3f757"}
Sep 16 18:42:33.828124 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:33.828072    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:42:33.850409 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:33.850371    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt" podStartSLOduration=1.8503595819999998 podStartE2EDuration="1.850359582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:33.850090968 +0000 UTC m=+1574.089879722" watchObservedRunningTime="2026-09-16 18:42:33.850359582 +0000 UTC m=+1574.090148333"
Sep 16 18:42:44.835141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:44.835112    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:42:55.408354 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:55.408308    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:56.195934 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:56.195880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:56.307933 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:56.307899    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:56.816124 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:56.816084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:57.162786 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:57.162748    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:57.244792 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:57.244766    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:57.677104 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:57.677073    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:58.391226 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:42:58.391196    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:00.085212 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:00.085179    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:00.518553 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:00.518518    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:00.617012 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:00.616978    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:01.140538 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:01.140509    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:02.239817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:02.239770    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:03.085716 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:03.085688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:03.901619 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:03.901556    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:04.002199 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.002176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:04.103116 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.103091    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:04.451243 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.451188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:04.537835 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.537804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:04.552171 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.552150    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"]
Sep 16 18:43:04.555989 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.555973    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:43:04.559923 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.559897    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-61fa84\""
Sep 16 18:43:04.560048 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.560032    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-61fa84-dockercfg-d868m\""
Sep 16 18:43:04.568467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.568446    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"]
Sep 16 18:43:04.669685 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.669666    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp6q9\" (UniqueName: \"kubernetes.io/projected/305525bc-1cb2-44cb-951b-f6cca0981ea2-kube-api-access-hp6q9\") pod \"payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb\" (UID: \"305525bc-1cb2-44cb-951b-f6cca0981ea2\") " pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:43:04.669790 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.669700    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/305525bc-1cb2-44cb-951b-f6cca0981ea2-plugins-config-volume\") pod \"payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb\" (UID: \"305525bc-1cb2-44cb-951b-f6cca0981ea2\") " pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:43:04.771194 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.771133    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/305525bc-1cb2-44cb-951b-f6cca0981ea2-plugins-config-volume\") pod \"payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb\" (UID: \"305525bc-1cb2-44cb-951b-f6cca0981ea2\") " pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:43:04.771303 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.771282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp6q9\" (UniqueName: \"kubernetes.io/projected/305525bc-1cb2-44cb-951b-f6cca0981ea2-kube-api-access-hp6q9\") pod \"payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb\" (UID: \"305525bc-1cb2-44cb-951b-f6cca0981ea2\") " pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:43:04.771831 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.771811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/305525bc-1cb2-44cb-951b-f6cca0981ea2-plugins-config-volume\") pod \"payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb\" (UID: \"305525bc-1cb2-44cb-951b-f6cca0981ea2\") " pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:43:04.780824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.780805    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp6q9\" (UniqueName: \"kubernetes.io/projected/305525bc-1cb2-44cb-951b-f6cca0981ea2-kube-api-access-hp6q9\") pod \"payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb\" (UID: \"305525bc-1cb2-44cb-951b-f6cca0981ea2\") " pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:43:04.865265 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.865242    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:43:04.997802 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:04.997778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:05.140929 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:05.140904    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"]
Sep 16 18:43:05.142263 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:43:05.142237    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod305525bc_1cb2_44cb_951b_f6cca0981ea2.slice/crio-524c84325754e0f0057f5612602e5b1333ecd7e2c3308c5d2f51c34107438e22 WatchSource:0}: Error finding container 524c84325754e0f0057f5612602e5b1333ecd7e2c3308c5d2f51c34107438e22: Status 404 returned error can't find the container with id 524c84325754e0f0057f5612602e5b1333ecd7e2c3308c5d2f51c34107438e22
Sep 16 18:43:05.802936 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:05.802904    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:05.997233 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:05.997204    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb" event={"ID":"305525bc-1cb2-44cb-951b-f6cca0981ea2","Type":"ContainerStarted","Data":"8440d40a8723448e3c7a3c441384025f0b468ff8f3547e2ad2d7c352fa149adf"}
Sep 16 18:43:05.997402 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:05.997251    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb" event={"ID":"305525bc-1cb2-44cb-951b-f6cca0981ea2","Type":"ContainerStarted","Data":"524c84325754e0f0057f5612602e5b1333ecd7e2c3308c5d2f51c34107438e22"}
Sep 16 18:43:05.997402 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:05.997299    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:43:06.025659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:06.025612    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb" podStartSLOduration=2.025583437 podStartE2EDuration="2.025583437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:43:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:43:06.023691714 +0000 UTC m=+1606.263480466" watchObservedRunningTime="2026-09-16 18:43:06.025583437 +0000 UTC m=+1606.265372188"
Sep 16 18:43:06.626017 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:06.625962    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:07.769153 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:07.769113    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:08.725058 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:08.725029    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:09.492383 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:09.492352    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:17.004668 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:17.004637    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:43:20.528141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:20.528113    2562 scope.go:117] "RemoveContainer" containerID="ff11590f58a17e378926294ab83580db47c06720c013f9492d3468fcdcc707a0"
Sep 16 18:43:20.536444 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:20.536419    2562 scope.go:117] "RemoveContainer" containerID="d4dbc0c62c4507d07b7de2ab8b68b6c21e3abf6639b73cb3d21aed70e33ee835"
Sep 16 18:43:22.694851 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:22.694810    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:23.553742 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:23.553713    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:23.671207 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:23.671178    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"]
Sep 16 18:43:23.674673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:23.674657    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"
Sep 16 18:43:23.677106 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:23.677086    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:43:23.677216 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:23.677163    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:43:23.684265 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:23.684244    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"]
Sep 16 18:43:23.735537 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:23.735515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b209fa7-9ae0-48d4-930f-03365d9e024c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2\" (UID: \"5b209fa7-9ae0-48d4-930f-03365d9e024c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"
Sep 16 18:43:23.835901 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:23.835838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b209fa7-9ae0-48d4-930f-03365d9e024c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2\" (UID: \"5b209fa7-9ae0-48d4-930f-03365d9e024c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"
Sep 16 18:43:23.836425 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:23.836408    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b209fa7-9ae0-48d4-930f-03365d9e024c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2\" (UID: \"5b209fa7-9ae0-48d4-930f-03365d9e024c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"
Sep 16 18:43:23.985522 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:23.985501    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"
Sep 16 18:43:24.006659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:24.006617    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:24.256778 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:24.256753    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"]
Sep 16 18:43:24.260283 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:43:24.260258    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b209fa7_9ae0_48d4_930f_03365d9e024c.slice/crio-90a261bfb07962aa4e7c69bd55c048e463edffd780f8f12f93f7b384eb6e0238 WatchSource:0}: Error finding container 90a261bfb07962aa4e7c69bd55c048e463edffd780f8f12f93f7b384eb6e0238: Status 404 returned error can't find the container with id 90a261bfb07962aa4e7c69bd55c048e463edffd780f8f12f93f7b384eb6e0238
Sep 16 18:43:25.002932 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:25.002903    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:25.096139 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:25.096081    2562 generic.go:356] "Generic (PLEG): container finished" podID="5b209fa7-9ae0-48d4-930f-03365d9e024c" containerID="eaad01e09b71cf6947c13f877e9d9322e554e09a166e8c49b79d01faaca74a8b" exitCode=0
Sep 16 18:43:25.096237 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:25.096165    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2" event={"ID":"5b209fa7-9ae0-48d4-930f-03365d9e024c","Type":"ContainerDied","Data":"eaad01e09b71cf6947c13f877e9d9322e554e09a166e8c49b79d01faaca74a8b"}
Sep 16 18:43:25.096237 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:25.096215    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2" event={"ID":"5b209fa7-9ae0-48d4-930f-03365d9e024c","Type":"ContainerStarted","Data":"90a261bfb07962aa4e7c69bd55c048e463edffd780f8f12f93f7b384eb6e0238"}
Sep 16 18:43:25.824756 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:25.824724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:26.239649 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:26.239625    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"
Sep 16 18:43:26.355858 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:26.355795    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b209fa7-9ae0-48d4-930f-03365d9e024c-maas-api-service-ca\") pod \"5b209fa7-9ae0-48d4-930f-03365d9e024c\" (UID: \"5b209fa7-9ae0-48d4-930f-03365d9e024c\") "
Sep 16 18:43:26.356101 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:26.356082    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b209fa7-9ae0-48d4-930f-03365d9e024c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5b209fa7-9ae0-48d4-930f-03365d9e024c" (UID: "5b209fa7-9ae0-48d4-930f-03365d9e024c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:43:26.457032 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:26.457009    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b209fa7-9ae0-48d4-930f-03365d9e024c-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:26.666727 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:26.666687    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:27.108781 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:27.108717    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2" event={"ID":"5b209fa7-9ae0-48d4-930f-03365d9e024c","Type":"ContainerDied","Data":"90a261bfb07962aa4e7c69bd55c048e463edffd780f8f12f93f7b384eb6e0238"}
Sep 16 18:43:27.108781 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:27.108733    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"
Sep 16 18:43:27.108781 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:27.108760    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90a261bfb07962aa4e7c69bd55c048e463edffd780f8f12f93f7b384eb6e0238"
Sep 16 18:43:27.620057 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:27.620023    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:28.554467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:28.554442    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"]
Sep 16 18:43:28.594378 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:28.594355    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"]
Sep 16 18:43:28.675062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:28.675040    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:28.775431 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:28.775408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:29.308985 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:29.308949    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:29.676495 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:29.676461    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:30.600976 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:30.600942    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:30.702209 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:30.702167    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:31.434803 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:31.434771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:32.378201 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:32.378164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:33.206887 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.206848    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:33.586907 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.586841    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"]
Sep 16 18:43:33.587280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.587266    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b209fa7-9ae0-48d4-930f-03365d9e024c" containerName="revoke-keys"
Sep 16 18:43:33.587280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.587280    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b209fa7-9ae0-48d4-930f-03365d9e024c" containerName="revoke-keys"
Sep 16 18:43:33.587364 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.587358    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b209fa7-9ae0-48d4-930f-03365d9e024c" containerName="revoke-keys"
Sep 16 18:43:33.590650 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.590636    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"
Sep 16 18:43:33.592974 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.592953    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:43:33.593100 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.593008    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:43:33.599378 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.599353    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"]
Sep 16 18:43:33.716363 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.716326    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7218ff29-626a-4e0f-bcc8-d29b27e5813a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr\" (UID: \"7218ff29-626a-4e0f-bcc8-d29b27e5813a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"
Sep 16 18:43:33.817680 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.817652    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7218ff29-626a-4e0f-bcc8-d29b27e5813a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr\" (UID: \"7218ff29-626a-4e0f-bcc8-d29b27e5813a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"
Sep 16 18:43:33.818201 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.818187    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7218ff29-626a-4e0f-bcc8-d29b27e5813a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr\" (UID: \"7218ff29-626a-4e0f-bcc8-d29b27e5813a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"
Sep 16 18:43:33.901483 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:33.901437    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"
Sep 16 18:43:34.101518 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:34.101490    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:34.173024 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:34.172998    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"]
Sep 16 18:43:34.174561 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:43:34.174525    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7218ff29_626a_4e0f_bcc8_d29b27e5813a.slice/crio-668d0a964fdf0ed9c12db4f6a658b63e2c4436c8fc39e535ae7757d2bba20ca1 WatchSource:0}: Error finding container 668d0a964fdf0ed9c12db4f6a658b63e2c4436c8fc39e535ae7757d2bba20ca1: Status 404 returned error can't find the container with id 668d0a964fdf0ed9c12db4f6a658b63e2c4436c8fc39e535ae7757d2bba20ca1
Sep 16 18:43:35.154367 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:35.154299    2562 generic.go:356] "Generic (PLEG): container finished" podID="7218ff29-626a-4e0f-bcc8-d29b27e5813a" containerID="e58402f757c5a74143821b95e8da425f71959f5ee0f555592e556459c9e2dfe3" exitCode=0
Sep 16 18:43:35.154750 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:35.154385    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr" event={"ID":"7218ff29-626a-4e0f-bcc8-d29b27e5813a","Type":"ContainerDied","Data":"e58402f757c5a74143821b95e8da425f71959f5ee0f555592e556459c9e2dfe3"}
Sep 16 18:43:35.154750 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:35.154481    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr" event={"ID":"7218ff29-626a-4e0f-bcc8-d29b27e5813a","Type":"ContainerStarted","Data":"668d0a964fdf0ed9c12db4f6a658b63e2c4436c8fc39e535ae7757d2bba20ca1"}
Sep 16 18:43:35.202752 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:35.202706    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:35.985841 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:35.985790    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:36.293018 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:36.292997    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"
Sep 16 18:43:36.438872 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:36.438848    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7218ff29-626a-4e0f-bcc8-d29b27e5813a-maas-api-service-ca\") pod \"7218ff29-626a-4e0f-bcc8-d29b27e5813a\" (UID: \"7218ff29-626a-4e0f-bcc8-d29b27e5813a\") "
Sep 16 18:43:36.439170 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:36.439150    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7218ff29-626a-4e0f-bcc8-d29b27e5813a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7218ff29-626a-4e0f-bcc8-d29b27e5813a" (UID: "7218ff29-626a-4e0f-bcc8-d29b27e5813a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:43:36.439249 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:36.439217    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7218ff29-626a-4e0f-bcc8-d29b27e5813a-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:36.661497 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:36.661469    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:37.165759 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:37.165737    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"
Sep 16 18:43:37.165905 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:37.165801    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr" event={"ID":"7218ff29-626a-4e0f-bcc8-d29b27e5813a","Type":"ContainerDied","Data":"668d0a964fdf0ed9c12db4f6a658b63e2c4436c8fc39e535ae7757d2bba20ca1"}
Sep 16 18:43:37.165905 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:37.165840    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="668d0a964fdf0ed9c12db4f6a658b63e2c4436c8fc39e535ae7757d2bba20ca1"
Sep 16 18:43:37.390613 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:37.390562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:38.105043 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:38.105016    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:39.177803 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:39.177764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:40.112640 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:40.112610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:43.573526 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.573491    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx" podUID="f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" containerName="main" containerID="cri-o://50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661" gracePeriod=45
Sep 16 18:43:43.612856 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.612826    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn" podUID="05112bb4-8288-4eb5-84ab-8e901a922f66" containerName="main" containerID="cri-o://1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26" gracePeriod=45
Sep 16 18:43:43.727878 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.727852    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:43.903464 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.903412    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:43:43.906870 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.906848    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:43:43.997259 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997234    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-home\") pod \"05112bb4-8288-4eb5-84ab-8e901a922f66\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") "
Sep 16 18:43:43.997259 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997262    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tmp-dir\") pod \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") "
Sep 16 18:43:43.997482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997300    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r82wf\" (UniqueName: \"kubernetes.io/projected/05112bb4-8288-4eb5-84ab-8e901a922f66-kube-api-access-r82wf\") pod \"05112bb4-8288-4eb5-84ab-8e901a922f66\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") "
Sep 16 18:43:43.997482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997330    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-model-cache\") pod \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") "
Sep 16 18:43:43.997482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997359    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-model-cache\") pod \"05112bb4-8288-4eb5-84ab-8e901a922f66\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") "
Sep 16 18:43:43.997482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997386    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-home\") pod \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") "
Sep 16 18:43:43.997482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997423    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-dshm\") pod \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") "
Sep 16 18:43:43.997482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997462    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05112bb4-8288-4eb5-84ab-8e901a922f66-tls-certs\") pod \"05112bb4-8288-4eb5-84ab-8e901a922f66\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") "
Sep 16 18:43:43.997832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997487    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwqvz\" (UniqueName: \"kubernetes.io/projected/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-kube-api-access-lwqvz\") pod \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") "
Sep 16 18:43:43.997832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997532    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tls-certs\") pod \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\" (UID: \"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e\") "
Sep 16 18:43:43.997832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997534    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-home" (OuterVolumeSpecName: "home") pod "05112bb4-8288-4eb5-84ab-8e901a922f66" (UID: "05112bb4-8288-4eb5-84ab-8e901a922f66"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:43:43.997832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997540    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" (UID: "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:43:43.997832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997647    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-dshm\") pod \"05112bb4-8288-4eb5-84ab-8e901a922f66\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") "
Sep 16 18:43:43.997832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997658    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-model-cache" (OuterVolumeSpecName: "model-cache") pod "05112bb4-8288-4eb5-84ab-8e901a922f66" (UID: "05112bb4-8288-4eb5-84ab-8e901a922f66"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:43:43.997832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997697    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-tmp-dir\") pod \"05112bb4-8288-4eb5-84ab-8e901a922f66\" (UID: \"05112bb4-8288-4eb5-84ab-8e901a922f66\") "
Sep 16 18:43:43.997832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997788    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-home" (OuterVolumeSpecName: "home") pod "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" (UID: "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:43:43.998215 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.997857    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-model-cache" (OuterVolumeSpecName: "model-cache") pod "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" (UID: "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:43:43.998215 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.998050    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "05112bb4-8288-4eb5-84ab-8e901a922f66" (UID: "05112bb4-8288-4eb5-84ab-8e901a922f66"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:43:43.998215 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.998177    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:43.998215 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.998195    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:43.998215 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.998208    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:43.998497 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.998220    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:43.998497 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.998275    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:43.998497 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:43.998306    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:44.000429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.000395    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05112bb4-8288-4eb5-84ab-8e901a922f66-kube-api-access-r82wf" (OuterVolumeSpecName: "kube-api-access-r82wf") pod "05112bb4-8288-4eb5-84ab-8e901a922f66" (UID: "05112bb4-8288-4eb5-84ab-8e901a922f66"). InnerVolumeSpecName "kube-api-access-r82wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:43:44.000702 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.000647    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-dshm" (OuterVolumeSpecName: "dshm") pod "05112bb4-8288-4eb5-84ab-8e901a922f66" (UID: "05112bb4-8288-4eb5-84ab-8e901a922f66"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:43:44.000702 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.000653    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-dshm" (OuterVolumeSpecName: "dshm") pod "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" (UID: "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:43:44.000702 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.000662    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05112bb4-8288-4eb5-84ab-8e901a922f66-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "05112bb4-8288-4eb5-84ab-8e901a922f66" (UID: "05112bb4-8288-4eb5-84ab-8e901a922f66"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:43:44.000702 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.000652    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" (UID: "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:43:44.000930 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.000772    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-kube-api-access-lwqvz" (OuterVolumeSpecName: "kube-api-access-lwqvz") pod "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" (UID: "f9e8c23c-fd1c-4e01-8d74-d9c37584de9e"). InnerVolumeSpecName "kube-api-access-lwqvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:43:44.099685 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.099661    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r82wf\" (UniqueName: \"kubernetes.io/projected/05112bb4-8288-4eb5-84ab-8e901a922f66-kube-api-access-r82wf\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:44.099685 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.099681    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:44.099817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.099691    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05112bb4-8288-4eb5-84ab-8e901a922f66-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:44.099817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.099700    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwqvz\" (UniqueName: \"kubernetes.io/projected/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-kube-api-access-lwqvz\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:44.099817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.099708    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:44.099817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.099716    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05112bb4-8288-4eb5-84ab-8e901a922f66-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:44.204430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.204400    2562 generic.go:356] "Generic (PLEG): container finished" podID="f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" containerID="50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661" exitCode=0
Sep 16 18:43:44.204544 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.204447    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"
Sep 16 18:43:44.204544 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.204486    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx" event={"ID":"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e","Type":"ContainerDied","Data":"50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661"}
Sep 16 18:43:44.204544 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.204527    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx" event={"ID":"f9e8c23c-fd1c-4e01-8d74-d9c37584de9e","Type":"ContainerDied","Data":"25acad902fe41b2161f95fadfc077aa88b7174432620bddf5a8dae18d43b1ff9"}
Sep 16 18:43:44.204758 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.204548    2562 scope.go:117] "RemoveContainer" containerID="50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661"
Sep 16 18:43:44.205878 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.205773    2562 generic.go:356] "Generic (PLEG): container finished" podID="05112bb4-8288-4eb5-84ab-8e901a922f66" containerID="1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26" exitCode=0
Sep 16 18:43:44.205878 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.205804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn" event={"ID":"05112bb4-8288-4eb5-84ab-8e901a922f66","Type":"ContainerDied","Data":"1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26"}
Sep 16 18:43:44.205878 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.205815    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"
Sep 16 18:43:44.205878 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.205832    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn" event={"ID":"05112bb4-8288-4eb5-84ab-8e901a922f66","Type":"ContainerDied","Data":"ab9fc634d902fdbc8a4072a1b3c813c0167ec08b9f18bc08c0c7471b70196ce7"}
Sep 16 18:43:44.214923 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.214908    2562 scope.go:117] "RemoveContainer" containerID="50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661"
Sep 16 18:43:44.215195 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:43:44.215177    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661\": container with ID starting with 50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661 not found: ID does not exist" containerID="50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661"
Sep 16 18:43:44.215241 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.215202    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661"} err="failed to get container status \"50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661\": rpc error: code = NotFound desc = could not find container \"50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661\": container with ID starting with 50ca4ea7e393b6c8868033767639f67a4ffe5ad35ff440fa6112bd1e66233661 not found: ID does not exist"
Sep 16 18:43:44.215241 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.215216    2562 scope.go:117] "RemoveContainer" containerID="1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26"
Sep 16 18:43:44.225570 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.225552    2562 scope.go:117] "RemoveContainer" containerID="1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26"
Sep 16 18:43:44.226100 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:43:44.226063    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26\": container with ID starting with 1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26 not found: ID does not exist" containerID="1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26"
Sep 16 18:43:44.226184 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.226098    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26"} err="failed to get container status \"1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26\": rpc error: code = NotFound desc = could not find container \"1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26\": container with ID starting with 1c31f2b41800c3d5896cb97969daa53c1923f72085114edef2df3c07559b1e26 not found: ID does not exist"
Sep 16 18:43:44.228215 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.228197    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"]
Sep 16 18:43:44.233196 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.233172    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/simulator-w0-a24d09-kserve-766c8ffd5c-87tsn"]
Sep 16 18:43:44.243008 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.242991    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"]
Sep 16 18:43:44.246710 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.246689    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-a24d09/distinct-w0-a24d09-kserve-6d6968f4bc-7mfvx"]
Sep 16 18:43:44.286003 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.285980    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05112bb4-8288-4eb5-84ab-8e901a922f66" path="/var/lib/kubelet/pods/05112bb4-8288-4eb5-84ab-8e901a922f66/volumes"
Sep 16 18:43:44.286332 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.286320    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" path="/var/lib/kubelet/pods/f9e8c23c-fd1c-4e01-8d74-d9c37584de9e/volumes"
Sep 16 18:43:44.344129 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.344100    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:44.383738 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.383708    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"]
Sep 16 18:43:44.383927 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.383902    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk" podUID="b5b272f8-ce53-48c4-ac86-55a90ee5e4b4" containerName="payload-pre-processing" containerID="cri-o://31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9" gracePeriod=30
Sep 16 18:43:44.459272 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.459201    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:44.557808 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.557775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:44.637488 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.637464    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:43:44.681976 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.681945    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:44.724344 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.724285    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"]
Sep 16 18:43:44.730544 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.730525    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-a24d09-g6ws2"]
Sep 16 18:43:44.767230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.767199    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:44.808453 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.808433    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mwhn\" (UniqueName: \"kubernetes.io/projected/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-kube-api-access-5mwhn\") pod \"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4\" (UID: \"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4\") "
Sep 16 18:43:44.808529 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.808492    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-plugins-config-volume\") pod \"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4\" (UID: \"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4\") "
Sep 16 18:43:44.808923 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.808894    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5b272f8-ce53-48c4-ac86-55a90ee5e4b4" (UID: "b5b272f8-ce53-48c4-ac86-55a90ee5e4b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:43:44.810563 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.810533    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-kube-api-access-5mwhn" (OuterVolumeSpecName: "kube-api-access-5mwhn") pod "b5b272f8-ce53-48c4-ac86-55a90ee5e4b4" (UID: "b5b272f8-ce53-48c4-ac86-55a90ee5e4b4"). InnerVolumeSpecName "kube-api-access-5mwhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:43:44.910145 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.910121    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:44.910145 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:44.910143    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mwhn\" (UniqueName: \"kubernetes.io/projected/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4-kube-api-access-5mwhn\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:45.090652 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:45.090577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:45.196528 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:45.196500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:45.211864 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:45.211841    2562 generic.go:356] "Generic (PLEG): container finished" podID="b5b272f8-ce53-48c4-ac86-55a90ee5e4b4" containerID="31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9" exitCode=0
Sep 16 18:43:45.211964 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:45.211894    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"
Sep 16 18:43:45.211964 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:45.211924    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk" event={"ID":"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4","Type":"ContainerDied","Data":"31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9"}
Sep 16 18:43:45.212060 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:45.212000    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk" event={"ID":"b5b272f8-ce53-48c4-ac86-55a90ee5e4b4","Type":"ContainerDied","Data":"c3949eade79bd139602f3fdd7b76246f426c0a1dcaf1452b43e31f0771d8df04"}
Sep 16 18:43:45.212060 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:45.212028    2562 scope.go:117] "RemoveContainer" containerID="31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9"
Sep 16 18:43:45.221519 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:45.221500    2562 scope.go:117] "RemoveContainer" containerID="31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9"
Sep 16 18:43:45.221788 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:43:45.221771    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9\": container with ID starting with 31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9 not found: ID does not exist" containerID="31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9"
Sep 16 18:43:45.221857 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:45.221795    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9"} err="failed to get container status \"31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9\": rpc error: code = NotFound desc = could not find container \"31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9\": container with ID starting with 31f8a81f72b5f9144dc70de7c898ce951c00abc2b189707b530a7b5dd41b99a9 not found: ID does not exist"
Sep 16 18:43:45.235766 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:45.235742    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"]
Sep 16 18:43:45.239674 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:45.239656    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-a24d09-557bbd68bb-2zjpk"]
Sep 16 18:43:46.013696 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:46.013668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:46.285100 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:46.285036    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b209fa7-9ae0-48d4-930f-03365d9e024c" path="/var/lib/kubelet/pods/5b209fa7-9ae0-48d4-930f-03365d9e024c/volumes"
Sep 16 18:43:46.285371 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:46.285359    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5b272f8-ce53-48c4-ac86-55a90ee5e4b4" path="/var/lib/kubelet/pods/b5b272f8-ce53-48c4-ac86-55a90ee5e4b4/volumes"
Sep 16 18:43:46.882387 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:46.882363    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:47.797836 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:47.797807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:48.605892 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:48.605855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:49.318053 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:49.318028    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:50.167998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:50.167968    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:50.802139 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:50.802112    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:50.834714 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:50.834687    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"]
Sep 16 18:43:50.834963 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:50.834937    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt" podUID="941ab0a7-e2b9-4a22-888a-69bc0e19cb6f" containerName="payload-processing" containerID="cri-o://0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103" gracePeriod=30
Sep 16 18:43:50.898747 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:50.898712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:51.000556 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.000522    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:51.002208 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.002156    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"]
Sep 16 18:43:51.004722 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.004685    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6a7741-s9hbr"]
Sep 16 18:43:51.082001 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.081976    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:43:51.102436 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.102411    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:51.159830 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.159809    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqr7q\" (UniqueName: \"kubernetes.io/projected/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-kube-api-access-vqr7q\") pod \"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f\" (UID: \"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f\") "
Sep 16 18:43:51.159954 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.159840    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-plugins-config-volume\") pod \"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f\" (UID: \"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f\") "
Sep 16 18:43:51.160234 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.160213    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "941ab0a7-e2b9-4a22-888a-69bc0e19cb6f" (UID: "941ab0a7-e2b9-4a22-888a-69bc0e19cb6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:43:51.161976 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.161954    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-kube-api-access-vqr7q" (OuterVolumeSpecName: "kube-api-access-vqr7q") pod "941ab0a7-e2b9-4a22-888a-69bc0e19cb6f" (UID: "941ab0a7-e2b9-4a22-888a-69bc0e19cb6f"). InnerVolumeSpecName "kube-api-access-vqr7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:43:51.247285 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.247254    2562 generic.go:356] "Generic (PLEG): container finished" podID="941ab0a7-e2b9-4a22-888a-69bc0e19cb6f" containerID="0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103" exitCode=0
Sep 16 18:43:51.247385 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.247300    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"
Sep 16 18:43:51.247385 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.247313    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt" event={"ID":"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f","Type":"ContainerDied","Data":"0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103"}
Sep 16 18:43:51.247385 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.247366    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt" event={"ID":"941ab0a7-e2b9-4a22-888a-69bc0e19cb6f","Type":"ContainerDied","Data":"d9c409583e8a78eb919c25ed5689a72b353740eb962fe809ffac6e873af3f757"}
Sep 16 18:43:51.247491 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.247395    2562 scope.go:117] "RemoveContainer" containerID="0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103"
Sep 16 18:43:51.256314 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.256296    2562 scope.go:117] "RemoveContainer" containerID="0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103"
Sep 16 18:43:51.256569 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:43:51.256547    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103\": container with ID starting with 0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103 not found: ID does not exist" containerID="0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103"
Sep 16 18:43:51.256683 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.256576    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103"} err="failed to get container status \"0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103\": rpc error: code = NotFound desc = could not find container \"0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103\": container with ID starting with 0cf6263568bda575281a7f8cc42b28fa03d667481b649bd58413380e26d5d103 not found: ID does not exist"
Sep 16 18:43:51.260631 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.260605    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqr7q\" (UniqueName: \"kubernetes.io/projected/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-kube-api-access-vqr7q\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:51.260701 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.260632    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:43:51.271328 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.271298    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"]
Sep 16 18:43:51.274716 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.274695    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6a7741-865bb5745f-m9ppt"]
Sep 16 18:43:51.377420 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.377364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:51.649369 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:51.649312    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:52.285893 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:52.285864    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7218ff29-626a-4e0f-bcc8-d29b27e5813a" path="/var/lib/kubelet/pods/7218ff29-626a-4e0f-bcc8-d29b27e5813a/volumes"
Sep 16 18:43:52.286229 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:52.286194    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="941ab0a7-e2b9-4a22-888a-69bc0e19cb6f" path="/var/lib/kubelet/pods/941ab0a7-e2b9-4a22-888a-69bc0e19cb6f/volumes"
Sep 16 18:43:52.419657 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:52.419623    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:53.178499 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:53.178474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:53.281843 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:53.281815    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:53.511201 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:53.511112    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:54.137701 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:54.137664    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:54.856120 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:54.856080    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:55.728160 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:55.728126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:56.455135 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:56.455099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:57.845216 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:57.845179    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:57.952269 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:57.952239    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:58.114180 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.114108    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:58.469695 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.469670    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"]
Sep 16 18:43:58.470125 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470108    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7218ff29-626a-4e0f-bcc8-d29b27e5813a" containerName="revoke-keys"
Sep 16 18:43:58.470218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470128    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7218ff29-626a-4e0f-bcc8-d29b27e5813a" containerName="revoke-keys"
Sep 16 18:43:58.470218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470148    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="941ab0a7-e2b9-4a22-888a-69bc0e19cb6f" containerName="payload-processing"
Sep 16 18:43:58.470218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470156    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="941ab0a7-e2b9-4a22-888a-69bc0e19cb6f" containerName="payload-processing"
Sep 16 18:43:58.470218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470171    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" containerName="main"
Sep 16 18:43:58.470218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470178    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" containerName="main"
Sep 16 18:43:58.470218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470196    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5b272f8-ce53-48c4-ac86-55a90ee5e4b4" containerName="payload-pre-processing"
Sep 16 18:43:58.470218 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470204    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5b272f8-ce53-48c4-ac86-55a90ee5e4b4" containerName="payload-pre-processing"
Sep 16 18:43:58.470779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470233    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05112bb4-8288-4eb5-84ab-8e901a922f66" containerName="main"
Sep 16 18:43:58.470779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470242    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="05112bb4-8288-4eb5-84ab-8e901a922f66" containerName="main"
Sep 16 18:43:58.470779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470336    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="05112bb4-8288-4eb5-84ab-8e901a922f66" containerName="main"
Sep 16 18:43:58.470779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470352    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7218ff29-626a-4e0f-bcc8-d29b27e5813a" containerName="revoke-keys"
Sep 16 18:43:58.470779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470361    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="941ab0a7-e2b9-4a22-888a-69bc0e19cb6f" containerName="payload-processing"
Sep 16 18:43:58.470779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470371    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5b272f8-ce53-48c4-ac86-55a90ee5e4b4" containerName="payload-pre-processing"
Sep 16 18:43:58.470779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.470379    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9e8c23c-fd1c-4e01-8d74-d9c37584de9e" containerName="main"
Sep 16 18:43:58.474826 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.474809    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"
Sep 16 18:43:58.477683 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.477666    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:43:58.477898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.477885    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:43:58.482067 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.482024    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"]
Sep 16 18:43:58.520294 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.520273    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dc65da0-5acc-45d3-8fc1-32208537f0ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-61fa84-6bkln\" (UID: \"9dc65da0-5acc-45d3-8fc1-32208537f0ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"
Sep 16 18:43:58.620857 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.620831    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dc65da0-5acc-45d3-8fc1-32208537f0ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-61fa84-6bkln\" (UID: \"9dc65da0-5acc-45d3-8fc1-32208537f0ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"
Sep 16 18:43:58.621361 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.621345    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dc65da0-5acc-45d3-8fc1-32208537f0ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-61fa84-6bkln\" (UID: \"9dc65da0-5acc-45d3-8fc1-32208537f0ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"
Sep 16 18:43:58.786629 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:58.786558    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"
Sep 16 18:43:59.068872 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:59.068846    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"]
Sep 16 18:43:59.070646 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:43:59.070622    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dc65da0_5acc_45d3_8fc1_32208537f0ea.slice/crio-f85f89553a442ddb7e0229b3afb8b436c9e21d797650bbeb5c7af9c333c9460d WatchSource:0}: Error finding container f85f89553a442ddb7e0229b3afb8b436c9e21d797650bbeb5c7af9c333c9460d: Status 404 returned error can't find the container with id f85f89553a442ddb7e0229b3afb8b436c9e21d797650bbeb5c7af9c333c9460d
Sep 16 18:43:59.295172 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:59.295140    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln" event={"ID":"9dc65da0-5acc-45d3-8fc1-32208537f0ea","Type":"ContainerStarted","Data":"f6894f03e1cb842b4767e6efc3a39f0a7369686a278ee92522477160f6495bbc"}
Sep 16 18:43:59.295293 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:59.295203    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln" event={"ID":"9dc65da0-5acc-45d3-8fc1-32208537f0ea","Type":"ContainerStarted","Data":"f85f89553a442ddb7e0229b3afb8b436c9e21d797650bbeb5c7af9c333c9460d"}
Sep 16 18:43:59.313426 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:59.313384    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln" podStartSLOduration=1.313370775 podStartE2EDuration="1.313370775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:43:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:43:59.311840566 +0000 UTC m=+1659.551629321" watchObservedRunningTime="2026-09-16 18:43:59.313370775 +0000 UTC m=+1659.553159527"
Sep 16 18:43:59.329536 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:43:59.329475    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:00.300754 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:00.300719    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:00.302241 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:00.302214    2562 generic.go:356] "Generic (PLEG): container finished" podID="9dc65da0-5acc-45d3-8fc1-32208537f0ea" containerID="f6894f03e1cb842b4767e6efc3a39f0a7369686a278ee92522477160f6495bbc" exitCode=0
Sep 16 18:44:00.302382 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:00.302298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln" event={"ID":"9dc65da0-5acc-45d3-8fc1-32208537f0ea","Type":"ContainerDied","Data":"f6894f03e1cb842b4767e6efc3a39f0a7369686a278ee92522477160f6495bbc"}
Sep 16 18:44:01.180206 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:01.180169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:01.442569 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:01.442517    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"
Sep 16 18:44:01.546420 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:01.546396    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dc65da0-5acc-45d3-8fc1-32208537f0ea-maas-api-service-ca\") pod \"9dc65da0-5acc-45d3-8fc1-32208537f0ea\" (UID: \"9dc65da0-5acc-45d3-8fc1-32208537f0ea\") "
Sep 16 18:44:01.546750 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:01.546730    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dc65da0-5acc-45d3-8fc1-32208537f0ea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9dc65da0-5acc-45d3-8fc1-32208537f0ea" (UID: "9dc65da0-5acc-45d3-8fc1-32208537f0ea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:44:01.647889 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:01.647865    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dc65da0-5acc-45d3-8fc1-32208537f0ea-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:44:02.212019 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:02.211992    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:02.314269 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:02.314242    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln" event={"ID":"9dc65da0-5acc-45d3-8fc1-32208537f0ea","Type":"ContainerDied","Data":"f85f89553a442ddb7e0229b3afb8b436c9e21d797650bbeb5c7af9c333c9460d"}
Sep 16 18:44:02.314386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:02.314292    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f85f89553a442ddb7e0229b3afb8b436c9e21d797650bbeb5c7af9c333c9460d"
Sep 16 18:44:02.314386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:02.314255    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"
Sep 16 18:44:02.962430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:02.962387    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:03.897359 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:03.897336    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:04.755053 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:04.754996    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:04.888388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:04.888355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:04.987827 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:04.987793    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:06.688771 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:06.688732    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:14.106011 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.105973    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:14.139483 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.139455    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"]
Sep 16 18:44:14.139732 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.139689    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb" podUID="305525bc-1cb2-44cb-951b-f6cca0981ea2" containerName="payload-processing" containerID="cri-o://8440d40a8723448e3c7a3c441384025f0b468ff8f3547e2ad2d7c352fa149adf" gracePeriod=30
Sep 16 18:44:14.183337 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.183314    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:14.284550 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.284521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:14.333782 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.333752    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"]
Sep 16 18:44:14.339636 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.339608    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61fa84-6bkln"]
Sep 16 18:44:14.353189 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.353164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:14.376087 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.376064    2562 generic.go:356] "Generic (PLEG): container finished" podID="305525bc-1cb2-44cb-951b-f6cca0981ea2" containerID="8440d40a8723448e3c7a3c441384025f0b468ff8f3547e2ad2d7c352fa149adf" exitCode=0
Sep 16 18:44:14.376198 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.376129    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb" event={"ID":"305525bc-1cb2-44cb-951b-f6cca0981ea2","Type":"ContainerDied","Data":"8440d40a8723448e3c7a3c441384025f0b468ff8f3547e2ad2d7c352fa149adf"}
Sep 16 18:44:14.376198 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.376164    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb" event={"ID":"305525bc-1cb2-44cb-951b-f6cca0981ea2","Type":"ContainerDied","Data":"524c84325754e0f0057f5612602e5b1333ecd7e2c3308c5d2f51c34107438e22"}
Sep 16 18:44:14.376198 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.376180    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="524c84325754e0f0057f5612602e5b1333ecd7e2c3308c5d2f51c34107438e22"
Sep 16 18:44:14.382476 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.382458    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:44:14.450521 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.450499    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/305525bc-1cb2-44cb-951b-f6cca0981ea2-plugins-config-volume\") pod \"305525bc-1cb2-44cb-951b-f6cca0981ea2\" (UID: \"305525bc-1cb2-44cb-951b-f6cca0981ea2\") "
Sep 16 18:44:14.450669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.450598    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp6q9\" (UniqueName: \"kubernetes.io/projected/305525bc-1cb2-44cb-951b-f6cca0981ea2-kube-api-access-hp6q9\") pod \"305525bc-1cb2-44cb-951b-f6cca0981ea2\" (UID: \"305525bc-1cb2-44cb-951b-f6cca0981ea2\") "
Sep 16 18:44:14.450830 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.450808    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/305525bc-1cb2-44cb-951b-f6cca0981ea2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "305525bc-1cb2-44cb-951b-f6cca0981ea2" (UID: "305525bc-1cb2-44cb-951b-f6cca0981ea2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:44:14.452663 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.452641    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/305525bc-1cb2-44cb-951b-f6cca0981ea2-kube-api-access-hp6q9" (OuterVolumeSpecName: "kube-api-access-hp6q9") pod "305525bc-1cb2-44cb-951b-f6cca0981ea2" (UID: "305525bc-1cb2-44cb-951b-f6cca0981ea2"). InnerVolumeSpecName "kube-api-access-hp6q9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:44:14.551969 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.551946    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/305525bc-1cb2-44cb-951b-f6cca0981ea2-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:44:14.551969 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.551967    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp6q9\" (UniqueName: \"kubernetes.io/projected/305525bc-1cb2-44cb-951b-f6cca0981ea2-kube-api-access-hp6q9\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:44:14.689375 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.689341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:14.817772 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:14.817740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:15.387003 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:15.386978    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"
Sep 16 18:44:15.408269 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:15.408246    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"]
Sep 16 18:44:15.412790 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:15.412761    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-61fa84-d5f794d69-l2fdb"]
Sep 16 18:44:15.501070 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:15.501050    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:16.285895 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:16.285870    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="305525bc-1cb2-44cb-951b-f6cca0981ea2" path="/var/lib/kubelet/pods/305525bc-1cb2-44cb-951b-f6cca0981ea2/volumes"
Sep 16 18:44:16.286191 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:16.286175    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dc65da0-5acc-45d3-8fc1-32208537f0ea" path="/var/lib/kubelet/pods/9dc65da0-5acc-45d3-8fc1-32208537f0ea/volumes"
Sep 16 18:44:16.292751 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:16.292719    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:17.164988 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:17.164954    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:17.890686 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:17.890655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:18.710416 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:18.710377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:21.113270 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.113233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:21.222365 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.222336    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:21.369512 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.369420    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:21.696393 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.696363    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"]
Sep 16 18:44:21.696766 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.696754    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dc65da0-5acc-45d3-8fc1-32208537f0ea" containerName="revoke-keys"
Sep 16 18:44:21.696817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.696768    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dc65da0-5acc-45d3-8fc1-32208537f0ea" containerName="revoke-keys"
Sep 16 18:44:21.696817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.696791    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="305525bc-1cb2-44cb-951b-f6cca0981ea2" containerName="payload-processing"
Sep 16 18:44:21.696817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.696796    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="305525bc-1cb2-44cb-951b-f6cca0981ea2" containerName="payload-processing"
Sep 16 18:44:21.696909 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.696861    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dc65da0-5acc-45d3-8fc1-32208537f0ea" containerName="revoke-keys"
Sep 16 18:44:21.696909 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.696875    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="305525bc-1cb2-44cb-951b-f6cca0981ea2" containerName="payload-processing"
Sep 16 18:44:21.699905 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.699890    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"
Sep 16 18:44:21.702337 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.702301    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:44:21.702636 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.702620    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:44:21.710443 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.710423    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"]
Sep 16 18:44:21.809942 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.809919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d055c111-eac3-462b-aebc-718c8581d63e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf\" (UID: \"d055c111-eac3-462b-aebc-718c8581d63e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"
Sep 16 18:44:21.910869 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.910845    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d055c111-eac3-462b-aebc-718c8581d63e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf\" (UID: \"d055c111-eac3-462b-aebc-718c8581d63e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"
Sep 16 18:44:21.911394 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:21.911378    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d055c111-eac3-462b-aebc-718c8581d63e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf\" (UID: \"d055c111-eac3-462b-aebc-718c8581d63e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"
Sep 16 18:44:22.010361 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:22.010309    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"
Sep 16 18:44:22.017939 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:22.017910    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:22.283163 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:44:22.283133    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd055c111_eac3_462b_aebc_718c8581d63e.slice/crio-20f2ca7f8c5a0db32b720c64c1a0789bcfda257cf78fc31ada70d64cc06367bc WatchSource:0}: Error finding container 20f2ca7f8c5a0db32b720c64c1a0789bcfda257cf78fc31ada70d64cc06367bc: Status 404 returned error can't find the container with id 20f2ca7f8c5a0db32b720c64c1a0789bcfda257cf78fc31ada70d64cc06367bc
Sep 16 18:44:22.287446 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:22.287421    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"]
Sep 16 18:44:22.425222 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:22.425195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf" event={"ID":"d055c111-eac3-462b-aebc-718c8581d63e","Type":"ContainerStarted","Data":"20f2ca7f8c5a0db32b720c64c1a0789bcfda257cf78fc31ada70d64cc06367bc"}
Sep 16 18:44:22.993066 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:22.993038    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:23.432202 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:23.432173    2562 generic.go:356] "Generic (PLEG): container finished" podID="d055c111-eac3-462b-aebc-718c8581d63e" containerID="970e45d7ea802f4fe89911e15b57fb58e5bfa0cabf516bd6e713c76350c1fdc0" exitCode=0
Sep 16 18:44:23.432503 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:23.432229    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf" event={"ID":"d055c111-eac3-462b-aebc-718c8581d63e","Type":"ContainerDied","Data":"970e45d7ea802f4fe89911e15b57fb58e5bfa0cabf516bd6e713c76350c1fdc0"}
Sep 16 18:44:23.977302 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:23.977270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:24.261180 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:24.261100    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:24.574377 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:24.574358    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"
Sep 16 18:44:24.732508 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:24.732482    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d055c111-eac3-462b-aebc-718c8581d63e-maas-api-service-ca\") pod \"d055c111-eac3-462b-aebc-718c8581d63e\" (UID: \"d055c111-eac3-462b-aebc-718c8581d63e\") "
Sep 16 18:44:24.732819 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:24.732797    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d055c111-eac3-462b-aebc-718c8581d63e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d055c111-eac3-462b-aebc-718c8581d63e" (UID: "d055c111-eac3-462b-aebc-718c8581d63e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:44:24.736350 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:24.736328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:24.833715 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:24.833661    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d055c111-eac3-462b-aebc-718c8581d63e-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:44:25.419113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:25.419081    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:25.443779 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:25.443758    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"
Sep 16 18:44:25.443894 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:25.443793    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf" event={"ID":"d055c111-eac3-462b-aebc-718c8581d63e","Type":"ContainerDied","Data":"20f2ca7f8c5a0db32b720c64c1a0789bcfda257cf78fc31ada70d64cc06367bc"}
Sep 16 18:44:25.443894 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:25.443832    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20f2ca7f8c5a0db32b720c64c1a0789bcfda257cf78fc31ada70d64cc06367bc"
Sep 16 18:44:26.225142 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:26.225116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:26.942049 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:26.942015    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:32.433110 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:32.433081    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:32.525479 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:32.525447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:32.626976 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:32.626941    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:32.655182 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:32.655158    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"]
Sep 16 18:44:32.657552 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:32.657530    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ad6826-9dhqf"]
Sep 16 18:44:32.724565 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:32.724505    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:33.036257 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:33.036193    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:33.226032 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:33.226004    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:34.030412 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:34.030383    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:34.286920 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:34.286860    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d055c111-eac3-462b-aebc-718c8581d63e" path="/var/lib/kubelet/pods/d055c111-eac3-462b-aebc-718c8581d63e/volumes"
Sep 16 18:44:34.913730 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:34.913702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:35.655422 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:35.655398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:36.366092 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:36.366065    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:39.805253 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:39.805224    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:39.869922 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:39.869892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:40.146247 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.146167    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:40.395016 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.394978    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"]
Sep 16 18:44:40.395390 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.395377    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d055c111-eac3-462b-aebc-718c8581d63e" containerName="revoke-keys"
Sep 16 18:44:40.395442 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.395392    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d055c111-eac3-462b-aebc-718c8581d63e" containerName="revoke-keys"
Sep 16 18:44:40.395481 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.395464    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d055c111-eac3-462b-aebc-718c8581d63e" containerName="revoke-keys"
Sep 16 18:44:40.398852 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.398813    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"
Sep 16 18:44:40.401118 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.401096    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:44:40.401225 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.401176    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:44:40.407942 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.407911    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"]
Sep 16 18:44:40.555017 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.554992    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:40.560096 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.560077    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3124ed73-35e8-4c9e-bbc3-35447901fbf1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l\" (UID: \"3124ed73-35e8-4c9e-bbc3-35447901fbf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"
Sep 16 18:44:40.661237 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.661217    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3124ed73-35e8-4c9e-bbc3-35447901fbf1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l\" (UID: \"3124ed73-35e8-4c9e-bbc3-35447901fbf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"
Sep 16 18:44:40.661786 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.661768    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3124ed73-35e8-4c9e-bbc3-35447901fbf1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l\" (UID: \"3124ed73-35e8-4c9e-bbc3-35447901fbf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"
Sep 16 18:44:40.709765 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.709743    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"
Sep 16 18:44:40.986456 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:40.986434    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"]
Sep 16 18:44:40.987978 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:44:40.987953    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3124ed73_35e8_4c9e_bbc3_35447901fbf1.slice/crio-5289b8610d84ca206b43925fab2e8301bb9f789f1b710e4aab23e95974dd2034 WatchSource:0}: Error finding container 5289b8610d84ca206b43925fab2e8301bb9f789f1b710e4aab23e95974dd2034: Status 404 returned error can't find the container with id 5289b8610d84ca206b43925fab2e8301bb9f789f1b710e4aab23e95974dd2034
Sep 16 18:44:41.525752 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:41.525678    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l" event={"ID":"3124ed73-35e8-4c9e-bbc3-35447901fbf1","Type":"ContainerStarted","Data":"cf1d87ef5a513ad2f3f42751f95b414bf9e3683284c0f388514ee08b657e7de3"}
Sep 16 18:44:41.525752 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:41.525723    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l" event={"ID":"3124ed73-35e8-4c9e-bbc3-35447901fbf1","Type":"ContainerStarted","Data":"5289b8610d84ca206b43925fab2e8301bb9f789f1b710e4aab23e95974dd2034"}
Sep 16 18:44:41.546934 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:41.546891    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l" podStartSLOduration=1.546877329 podStartE2EDuration="1.546877329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:44:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:41.544764776 +0000 UTC m=+1701.784553528" watchObservedRunningTime="2026-09-16 18:44:41.546877329 +0000 UTC m=+1701.786666129"
Sep 16 18:44:41.707901 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:41.707860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:42.448040 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:42.448006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:42.532386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:42.532362    2562 generic.go:356] "Generic (PLEG): container finished" podID="3124ed73-35e8-4c9e-bbc3-35447901fbf1" containerID="cf1d87ef5a513ad2f3f42751f95b414bf9e3683284c0f388514ee08b657e7de3" exitCode=0
Sep 16 18:44:42.532507 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:42.532443    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l" event={"ID":"3124ed73-35e8-4c9e-bbc3-35447901fbf1","Type":"ContainerDied","Data":"cf1d87ef5a513ad2f3f42751f95b414bf9e3683284c0f388514ee08b657e7de3"}
Sep 16 18:44:43.180695 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:43.180655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:43.665350 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:43.665330    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"
Sep 16 18:44:43.788557 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:43.788491    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3124ed73-35e8-4c9e-bbc3-35447901fbf1-maas-api-service-ca\") pod \"3124ed73-35e8-4c9e-bbc3-35447901fbf1\" (UID: \"3124ed73-35e8-4c9e-bbc3-35447901fbf1\") "
Sep 16 18:44:43.788835 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:43.788814    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3124ed73-35e8-4c9e-bbc3-35447901fbf1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3124ed73-35e8-4c9e-bbc3-35447901fbf1" (UID: "3124ed73-35e8-4c9e-bbc3-35447901fbf1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:44:43.889517 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:43.889491    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3124ed73-35e8-4c9e-bbc3-35447901fbf1-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:44:44.012291 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:44.012265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:44.544552 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:44.544501    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"
Sep 16 18:44:44.544552 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:44.544537    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l" event={"ID":"3124ed73-35e8-4c9e-bbc3-35447901fbf1","Type":"ContainerDied","Data":"5289b8610d84ca206b43925fab2e8301bb9f789f1b710e4aab23e95974dd2034"}
Sep 16 18:44:44.544745 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:44.544578    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5289b8610d84ca206b43925fab2e8301bb9f789f1b710e4aab23e95974dd2034"
Sep 16 18:44:44.827195 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:44.827134    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:48.883899 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:48.883863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:48.989475 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:48.989442    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"]
Sep 16 18:44:48.989917 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:48.989900    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3124ed73-35e8-4c9e-bbc3-35447901fbf1" containerName="revoke-keys"
Sep 16 18:44:48.990002 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:48.989920    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3124ed73-35e8-4c9e-bbc3-35447901fbf1" containerName="revoke-keys"
Sep 16 18:44:48.990063 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:48.990027    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3124ed73-35e8-4c9e-bbc3-35447901fbf1" containerName="revoke-keys"
Sep 16 18:44:48.993170 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:48.993153    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"
Sep 16 18:44:48.995619 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:48.995577    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:44:48.995619 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:48.995612    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:44:49.001286 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:49.001264    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"]
Sep 16 18:44:49.132971 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:49.132944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7565bd46-2997-4f3d-a62d-f6b6e2cc72ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294\" (UID: \"7565bd46-2997-4f3d-a62d-f6b6e2cc72ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"
Sep 16 18:44:49.234181 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:49.234157    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7565bd46-2997-4f3d-a62d-f6b6e2cc72ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294\" (UID: \"7565bd46-2997-4f3d-a62d-f6b6e2cc72ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"
Sep 16 18:44:49.234724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:49.234709    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7565bd46-2997-4f3d-a62d-f6b6e2cc72ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294\" (UID: \"7565bd46-2997-4f3d-a62d-f6b6e2cc72ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"
Sep 16 18:44:49.304583 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:49.304562    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"
Sep 16 18:44:49.480000 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:49.479972    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:49.578290 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:49.578267    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"]
Sep 16 18:44:49.580405 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:44:49.580377    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7565bd46_2997_4f3d_a62d_f6b6e2cc72ca.slice/crio-d5edecfe27ad3e2b585dc9445bc7915aa52c8072501099e82201121d08a569ea WatchSource:0}: Error finding container d5edecfe27ad3e2b585dc9445bc7915aa52c8072501099e82201121d08a569ea: Status 404 returned error can't find the container with id d5edecfe27ad3e2b585dc9445bc7915aa52c8072501099e82201121d08a569ea
Sep 16 18:44:50.278793 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:50.278760    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:50.577962 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:50.577936    2562 generic.go:356] "Generic (PLEG): container finished" podID="7565bd46-2997-4f3d-a62d-f6b6e2cc72ca" containerID="0788812be8c507bd06b28f0e37e34de9b52ab2da0bc0695ef24569e811c42a5d" exitCode=0
Sep 16 18:44:50.578095 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:50.578020    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294" event={"ID":"7565bd46-2997-4f3d-a62d-f6b6e2cc72ca","Type":"ContainerDied","Data":"0788812be8c507bd06b28f0e37e34de9b52ab2da0bc0695ef24569e811c42a5d"}
Sep 16 18:44:50.578095 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:50.578071    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294" event={"ID":"7565bd46-2997-4f3d-a62d-f6b6e2cc72ca","Type":"ContainerStarted","Data":"d5edecfe27ad3e2b585dc9445bc7915aa52c8072501099e82201121d08a569ea"}
Sep 16 18:44:51.096492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:51.096461    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:51.715414 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:51.715392    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"
Sep 16 18:44:51.752839 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:51.752814    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7565bd46-2997-4f3d-a62d-f6b6e2cc72ca-maas-api-service-ca\") pod \"7565bd46-2997-4f3d-a62d-f6b6e2cc72ca\" (UID: \"7565bd46-2997-4f3d-a62d-f6b6e2cc72ca\") "
Sep 16 18:44:51.753129 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:51.753109    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7565bd46-2997-4f3d-a62d-f6b6e2cc72ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7565bd46-2997-4f3d-a62d-f6b6e2cc72ca" (UID: "7565bd46-2997-4f3d-a62d-f6b6e2cc72ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:44:51.853529 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:51.853478    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7565bd46-2997-4f3d-a62d-f6b6e2cc72ca-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:44:51.899524 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:51.899489    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:52.590011 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:52.589948    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"
Sep 16 18:44:52.590011 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:52.589973    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294" event={"ID":"7565bd46-2997-4f3d-a62d-f6b6e2cc72ca","Type":"ContainerDied","Data":"d5edecfe27ad3e2b585dc9445bc7915aa52c8072501099e82201121d08a569ea"}
Sep 16 18:44:52.590172 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:52.590020    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5edecfe27ad3e2b585dc9445bc7915aa52c8072501099e82201121d08a569ea"
Sep 16 18:44:52.663030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:52.662994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:53.585182 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:53.585129    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:53.873485 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:53.873422    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"]
Sep 16 18:44:53.881388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:53.881364    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"]
Sep 16 18:44:53.892658 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:53.892620    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"]
Sep 16 18:44:53.973314 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:53.973288    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:54.302309 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:54.302277    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:54.376008 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:54.375974    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:54.455259 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:54.455230    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:54.508852 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:54.508825    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"]
Sep 16 18:44:54.512492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:54.512469    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-cf62d1-v8b6l"]
Sep 16 18:44:54.531502 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:54.531477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:54.844202 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:54.844176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:55.437555 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:55.437524    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:56.286298 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:56.286268    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3124ed73-35e8-4c9e-bbc3-35447901fbf1" path="/var/lib/kubelet/pods/3124ed73-35e8-4c9e-bbc3-35447901fbf1/volumes"
Sep 16 18:44:56.297801 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:56.297780    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:57.116082 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:57.116054    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:59.355692 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:44:59.355656    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:00.152666 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.152640    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn"]
Sep 16 18:45:00.153024 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.153012    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7565bd46-2997-4f3d-a62d-f6b6e2cc72ca" containerName="revoke-keys"
Sep 16 18:45:00.153082 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.153026    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7565bd46-2997-4f3d-a62d-f6b6e2cc72ca" containerName="revoke-keys"
Sep 16 18:45:00.153129 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.153121    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7565bd46-2997-4f3d-a62d-f6b6e2cc72ca" containerName="revoke-keys"
Sep 16 18:45:00.156277 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.156262    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn"
Sep 16 18:45:00.158777 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.158751    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:45:00.165947 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.165927    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826405-k8hzw"]
Sep 16 18:45:00.166263 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.166248    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn"
Sep 16 18:45:00.169246 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.169226    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826405-k8hzw"
Sep 16 18:45:00.172421 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.172400    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn"]
Sep 16 18:45:00.194125 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.194098    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826405-k8hzw"]
Sep 16 18:45:00.206357 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.206340    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826405-k8hzw"
Sep 16 18:45:00.460119 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.460096    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn"]
Sep 16 18:45:00.462022 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:45:00.461996    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62e2177b_d228_4107_8bb3_f8f0f7546d36.slice/crio-0eca91134cab3bac1ffbc46f3b9bcaa7804dd66d6f8450b911c0daccf834aa5b WatchSource:0}: Error finding container 0eca91134cab3bac1ffbc46f3b9bcaa7804dd66d6f8450b911c0daccf834aa5b: Status 404 returned error can't find the container with id 0eca91134cab3bac1ffbc46f3b9bcaa7804dd66d6f8450b911c0daccf834aa5b
Sep 16 18:45:00.502967 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.502947    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826405-k8hzw"]
Sep 16 18:45:00.503857 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:45:00.503834    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda094a321_6f0f_4719_af2c_d9dc6ca8bc24.slice/crio-673530d508c39727765dcd9a32025acf8141c8ecb4ef5c1614d4ff628fa3ea4e WatchSource:0}: Error finding container 673530d508c39727765dcd9a32025acf8141c8ecb4ef5c1614d4ff628fa3ea4e: Status 404 returned error can't find the container with id 673530d508c39727765dcd9a32025acf8141c8ecb4ef5c1614d4ff628fa3ea4e
Sep 16 18:45:00.639819 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.639796    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826405-k8hzw" event={"ID":"a094a321-6f0f-4719-af2c-d9dc6ca8bc24","Type":"ContainerStarted","Data":"673530d508c39727765dcd9a32025acf8141c8ecb4ef5c1614d4ff628fa3ea4e"}
Sep 16 18:45:00.641106 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:00.641084    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn" event={"ID":"62e2177b-d228-4107-8bb3-f8f0f7546d36","Type":"ContainerStarted","Data":"0eca91134cab3bac1ffbc46f3b9bcaa7804dd66d6f8450b911c0daccf834aa5b"}
Sep 16 18:45:01.648834 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:01.648759    2562 generic.go:356] "Generic (PLEG): container finished" podID="62e2177b-d228-4107-8bb3-f8f0f7546d36" containerID="55fd156e83a0a82b34566d99b01d2bb741ea4fb04ba6c85e41641efd0b966607" exitCode=0
Sep 16 18:45:01.649177 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:01.648846    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn" event={"ID":"62e2177b-d228-4107-8bb3-f8f0f7546d36","Type":"ContainerDied","Data":"55fd156e83a0a82b34566d99b01d2bb741ea4fb04ba6c85e41641efd0b966607"}
Sep 16 18:45:01.650426 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:01.650400    2562 generic.go:356] "Generic (PLEG): container finished" podID="a094a321-6f0f-4719-af2c-d9dc6ca8bc24" containerID="6d0f03be14e3cb9f6777114f7a675231d4507cf6244d24fb753ef644c94b1ebd" exitCode=0
Sep 16 18:45:01.650536 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:01.650458    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826405-k8hzw" event={"ID":"a094a321-6f0f-4719-af2c-d9dc6ca8bc24","Type":"ContainerDied","Data":"6d0f03be14e3cb9f6777114f7a675231d4507cf6244d24fb753ef644c94b1ebd"}
Sep 16 18:45:01.932954 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:01.932928    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:02.032772 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:02.032739    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:02.122228 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:02.122190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:02.800614 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:02.800567    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826405-k8hzw"
Sep 16 18:45:02.803182 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:02.803167    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn"
Sep 16 18:45:03.664317 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:03.664290    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn"
Sep 16 18:45:03.664486 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:03.664332    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn" event={"ID":"62e2177b-d228-4107-8bb3-f8f0f7546d36","Type":"ContainerDied","Data":"0eca91134cab3bac1ffbc46f3b9bcaa7804dd66d6f8450b911c0daccf834aa5b"}
Sep 16 18:45:03.664486 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:03.664378    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0eca91134cab3bac1ffbc46f3b9bcaa7804dd66d6f8450b911c0daccf834aa5b"
Sep 16 18:45:03.665818 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:03.665795    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826405-k8hzw" event={"ID":"a094a321-6f0f-4719-af2c-d9dc6ca8bc24","Type":"ContainerDied","Data":"673530d508c39727765dcd9a32025acf8141c8ecb4ef5c1614d4ff628fa3ea4e"}
Sep 16 18:45:03.665940 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:03.665822    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826405-k8hzw"
Sep 16 18:45:03.665940 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:03.665835    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="673530d508c39727765dcd9a32025acf8141c8ecb4ef5c1614d4ff628fa3ea4e"
Sep 16 18:45:05.127937 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.127906    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:05.140934 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.140905    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn"]
Sep 16 18:45:05.142874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.142856    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-6942a4-29826405-m7htn"]
Sep 16 18:45:05.167096 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.167070    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"]
Sep 16 18:45:05.167307 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.167279    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd" podUID="50d8f3f3-3f6b-4440-aaa5-0c16e8104c18" containerName="payload-processing" containerID="cri-o://5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3" gracePeriod=30
Sep 16 18:45:05.177396 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.177377    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"]
Sep 16 18:45:05.177609 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.177561    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm" podUID="9bf78544-63da-42b4-af9f-7dc6712ce018" containerName="payload-pre-processing" containerID="cri-o://b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840" gracePeriod=30
Sep 16 18:45:05.209626 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.209580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:05.279939 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.279904    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:05.281042 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:45:05.281018    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d8f3f3_3f6b_4440_aaa5_0c16e8104c18.slice/crio-conmon-5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 18:45:05.352356 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.351330    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"]
Sep 16 18:45:05.355358 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.355333    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-6942a4-qv294"]
Sep 16 18:45:05.355974 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.355956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:05.420055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.420033    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:45:05.428427 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.428393    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:05.461325 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.461300    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:45:05.462177 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.462158    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmrm4\" (UniqueName: \"kubernetes.io/projected/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-kube-api-access-nmrm4\") pod \"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18\" (UID: \"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18\") "
Sep 16 18:45:05.462259 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.462215    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-plugins-config-volume\") pod \"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18\" (UID: \"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18\") "
Sep 16 18:45:05.462599 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.462563    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "50d8f3f3-3f6b-4440-aaa5-0c16e8104c18" (UID: "50d8f3f3-3f6b-4440-aaa5-0c16e8104c18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:45:05.464395 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.464373    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-kube-api-access-nmrm4" (OuterVolumeSpecName: "kube-api-access-nmrm4") pod "50d8f3f3-3f6b-4440-aaa5-0c16e8104c18" (UID: "50d8f3f3-3f6b-4440-aaa5-0c16e8104c18"). InnerVolumeSpecName "kube-api-access-nmrm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:45:05.563826 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.563802    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h8vk\" (UniqueName: \"kubernetes.io/projected/9bf78544-63da-42b4-af9f-7dc6712ce018-kube-api-access-7h8vk\") pod \"9bf78544-63da-42b4-af9f-7dc6712ce018\" (UID: \"9bf78544-63da-42b4-af9f-7dc6712ce018\") "
Sep 16 18:45:05.563937 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.563887    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf78544-63da-42b4-af9f-7dc6712ce018-plugins-config-volume\") pod \"9bf78544-63da-42b4-af9f-7dc6712ce018\" (UID: \"9bf78544-63da-42b4-af9f-7dc6712ce018\") "
Sep 16 18:45:05.564083 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.564070    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmrm4\" (UniqueName: \"kubernetes.io/projected/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-kube-api-access-nmrm4\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:05.564083 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.564083    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:05.564260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.564238    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bf78544-63da-42b4-af9f-7dc6712ce018-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9bf78544-63da-42b4-af9f-7dc6712ce018" (UID: "9bf78544-63da-42b4-af9f-7dc6712ce018"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:45:05.565690 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.565670    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bf78544-63da-42b4-af9f-7dc6712ce018-kube-api-access-7h8vk" (OuterVolumeSpecName: "kube-api-access-7h8vk") pod "9bf78544-63da-42b4-af9f-7dc6712ce018" (UID: "9bf78544-63da-42b4-af9f-7dc6712ce018"). InnerVolumeSpecName "kube-api-access-7h8vk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:45:05.664548 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.664530    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf78544-63da-42b4-af9f-7dc6712ce018-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:05.664548 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.664547    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h8vk\" (UniqueName: \"kubernetes.io/projected/9bf78544-63da-42b4-af9f-7dc6712ce018-kube-api-access-7h8vk\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:05.677189 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.677168    2562 generic.go:356] "Generic (PLEG): container finished" podID="50d8f3f3-3f6b-4440-aaa5-0c16e8104c18" containerID="5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3" exitCode=0
Sep 16 18:45:05.677278 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.677215    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"
Sep 16 18:45:05.677278 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.677243    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd" event={"ID":"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18","Type":"ContainerDied","Data":"5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3"}
Sep 16 18:45:05.677385 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.677290    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd" event={"ID":"50d8f3f3-3f6b-4440-aaa5-0c16e8104c18","Type":"ContainerDied","Data":"a93e75893aedec90d62b54702b227b2c02c5624f684aa70341d2c5616c18a375"}
Sep 16 18:45:05.677385 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.677318    2562 scope.go:117] "RemoveContainer" containerID="5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3"
Sep 16 18:45:05.678814 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.678795    2562 generic.go:356] "Generic (PLEG): container finished" podID="9bf78544-63da-42b4-af9f-7dc6712ce018" containerID="b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840" exitCode=0
Sep 16 18:45:05.678922 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.678838    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"
Sep 16 18:45:05.678922 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.678861    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm" event={"ID":"9bf78544-63da-42b4-af9f-7dc6712ce018","Type":"ContainerDied","Data":"b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840"}
Sep 16 18:45:05.679039 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.678923    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm" event={"ID":"9bf78544-63da-42b4-af9f-7dc6712ce018","Type":"ContainerDied","Data":"a019ce300e09307eb731163c3115bd4496907712f97cc777d2dbd7534f56eb29"}
Sep 16 18:45:05.691844 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.691818    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:05.695174 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.695158    2562 scope.go:117] "RemoveContainer" containerID="5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3"
Sep 16 18:45:05.695419 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:45:05.695403    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3\": container with ID starting with 5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3 not found: ID does not exist" containerID="5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3"
Sep 16 18:45:05.695467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.695428    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3"} err="failed to get container status \"5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3\": rpc error: code = NotFound desc = could not find container \"5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3\": container with ID starting with 5f493eef0381b0cf70ca2b9bf0ef48ab05b3601f981ec0d20522f9c12222b5c3 not found: ID does not exist"
Sep 16 18:45:05.695467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.695449    2562 scope.go:117] "RemoveContainer" containerID="b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840"
Sep 16 18:45:05.703729 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.703713    2562 scope.go:117] "RemoveContainer" containerID="b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840"
Sep 16 18:45:05.703957 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:45:05.703942    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840\": container with ID starting with b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840 not found: ID does not exist" containerID="b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840"
Sep 16 18:45:05.704004 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.703962    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840"} err="failed to get container status \"b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840\": rpc error: code = NotFound desc = could not find container \"b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840\": container with ID starting with b025115142d2260a9508c42970e8de751bd78177e9fbf11d466d15a85bd59840 not found: ID does not exist"
Sep 16 18:45:05.708421 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.708403    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"]
Sep 16 18:45:05.711495 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.711475    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-6942a4-b8c5c9687-f92cd"]
Sep 16 18:45:05.721532 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.721513    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"]
Sep 16 18:45:05.725903 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.725885    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-6942a4-fb4b87c6b-fhrtm"]
Sep 16 18:45:05.830373 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:05.830350    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:06.285262 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:06.285237    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50d8f3f3-3f6b-4440-aaa5-0c16e8104c18" path="/var/lib/kubelet/pods/50d8f3f3-3f6b-4440-aaa5-0c16e8104c18/volumes"
Sep 16 18:45:06.285553 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:06.285493    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62e2177b-d228-4107-8bb3-f8f0f7546d36" path="/var/lib/kubelet/pods/62e2177b-d228-4107-8bb3-f8f0f7546d36/volumes"
Sep 16 18:45:06.285767 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:06.285754    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7565bd46-2997-4f3d-a62d-f6b6e2cc72ca" path="/var/lib/kubelet/pods/7565bd46-2997-4f3d-a62d-f6b6e2cc72ca/volumes"
Sep 16 18:45:06.286055 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:06.286044    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bf78544-63da-42b4-af9f-7dc6712ce018" path="/var/lib/kubelet/pods/9bf78544-63da-42b4-af9f-7dc6712ce018/volumes"
Sep 16 18:45:06.771552 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:06.771523    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:07.743430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:07.743390    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:08.893208 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:08.893178    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k" podUID="e718cdac-500a-478f-8e6d-3951cdd636b0" containerName="main" containerID="cri-o://1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46" gracePeriod=45
Sep 16 18:45:08.900128 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:08.900099    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb" podUID="8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" containerName="main" containerID="cri-o://e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944" gracePeriod=45
Sep 16 18:45:08.914961 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:08.914929    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2" podUID="de0b960c-ca89-472a-be3e-675ee166b0ed" containerName="main" containerID="cri-o://cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0" gracePeriod=45
Sep 16 18:45:09.178510 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.178481    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:45:09.204267 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.204231    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:45:09.207812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.207791    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:45:09.293967 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.293937    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e718cdac-500a-478f-8e6d-3951cdd636b0-tls-certs\") pod \"e718cdac-500a-478f-8e6d-3951cdd636b0\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") "
Sep 16 18:45:09.294141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.293980    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqjhb\" (UniqueName: \"kubernetes.io/projected/e718cdac-500a-478f-8e6d-3951cdd636b0-kube-api-access-gqjhb\") pod \"e718cdac-500a-478f-8e6d-3951cdd636b0\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") "
Sep 16 18:45:09.294141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294007    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-dshm\") pod \"de0b960c-ca89-472a-be3e-675ee166b0ed\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") "
Sep 16 18:45:09.294141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294027    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-home\") pod \"e718cdac-500a-478f-8e6d-3951cdd636b0\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") "
Sep 16 18:45:09.294141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294052    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lg926\" (UniqueName: \"kubernetes.io/projected/de0b960c-ca89-472a-be3e-675ee166b0ed-kube-api-access-lg926\") pod \"de0b960c-ca89-472a-be3e-675ee166b0ed\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") "
Sep 16 18:45:09.294141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294086    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-model-cache\") pod \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") "
Sep 16 18:45:09.294141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294112    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-tmp-dir\") pod \"de0b960c-ca89-472a-be3e-675ee166b0ed\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") "
Sep 16 18:45:09.294458 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294171    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tls-certs\") pod \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") "
Sep 16 18:45:09.294458 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294308    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-home" (OuterVolumeSpecName: "home") pod "e718cdac-500a-478f-8e6d-3951cdd636b0" (UID: "e718cdac-500a-478f-8e6d-3951cdd636b0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.294458 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294360    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-model-cache" (OuterVolumeSpecName: "model-cache") pod "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" (UID: "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.294458 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294427    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de0b960c-ca89-472a-be3e-675ee166b0ed" (UID: "de0b960c-ca89-472a-be3e-675ee166b0ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.294684 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294528    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-model-cache\") pod \"e718cdac-500a-478f-8e6d-3951cdd636b0\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") "
Sep 16 18:45:09.294684 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294573    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldnvw\" (UniqueName: \"kubernetes.io/projected/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-kube-api-access-ldnvw\") pod \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") "
Sep 16 18:45:09.294684 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294619    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-tmp-dir\") pod \"e718cdac-500a-478f-8e6d-3951cdd636b0\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") "
Sep 16 18:45:09.294684 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294663    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-model-cache\") pod \"de0b960c-ca89-472a-be3e-675ee166b0ed\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") "
Sep 16 18:45:09.294873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294693    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-dshm\") pod \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") "
Sep 16 18:45:09.294873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294717    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tmp-dir\") pod \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") "
Sep 16 18:45:09.294873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294816    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-model-cache" (OuterVolumeSpecName: "model-cache") pod "e718cdac-500a-478f-8e6d-3951cdd636b0" (UID: "e718cdac-500a-478f-8e6d-3951cdd636b0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.294873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294832    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de0b960c-ca89-472a-be3e-675ee166b0ed-tls-certs\") pod \"de0b960c-ca89-472a-be3e-675ee166b0ed\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") "
Sep 16 18:45:09.294873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294856    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-dshm\") pod \"e718cdac-500a-478f-8e6d-3951cdd636b0\" (UID: \"e718cdac-500a-478f-8e6d-3951cdd636b0\") "
Sep 16 18:45:09.295107 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294882    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-home\") pod \"de0b960c-ca89-472a-be3e-675ee166b0ed\" (UID: \"de0b960c-ca89-472a-be3e-675ee166b0ed\") "
Sep 16 18:45:09.295107 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294910    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-home\") pod \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\" (UID: \"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df\") "
Sep 16 18:45:09.295107 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.294994    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "de0b960c-ca89-472a-be3e-675ee166b0ed" (UID: "de0b960c-ca89-472a-be3e-675ee166b0ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.295235 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.295143    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e718cdac-500a-478f-8e6d-3951cdd636b0" (UID: "e718cdac-500a-478f-8e6d-3951cdd636b0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.295285 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.295271    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.295345 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.295290    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.295345 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.295302    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.295345 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.295314    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.295345 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.295326    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.295345 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.295338    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.297056 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.297026    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-home" (OuterVolumeSpecName: "home") pod "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" (UID: "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.297056 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.297028    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-home" (OuterVolumeSpecName: "home") pod "de0b960c-ca89-472a-be3e-675ee166b0ed" (UID: "de0b960c-ca89-472a-be3e-675ee166b0ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.297056 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.297079    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-dshm" (OuterVolumeSpecName: "dshm") pod "de0b960c-ca89-472a-be3e-675ee166b0ed" (UID: "de0b960c-ca89-472a-be3e-675ee166b0ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.297467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.297150    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e718cdac-500a-478f-8e6d-3951cdd636b0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e718cdac-500a-478f-8e6d-3951cdd636b0" (UID: "e718cdac-500a-478f-8e6d-3951cdd636b0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:45:09.297467 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.297344    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" (UID: "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.297834 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.297762    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e718cdac-500a-478f-8e6d-3951cdd636b0-kube-api-access-gqjhb" (OuterVolumeSpecName: "kube-api-access-gqjhb") pod "e718cdac-500a-478f-8e6d-3951cdd636b0" (UID: "e718cdac-500a-478f-8e6d-3951cdd636b0"). InnerVolumeSpecName "kube-api-access-gqjhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:45:09.297834 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.297778    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de0b960c-ca89-472a-be3e-675ee166b0ed-kube-api-access-lg926" (OuterVolumeSpecName: "kube-api-access-lg926") pod "de0b960c-ca89-472a-be3e-675ee166b0ed" (UID: "de0b960c-ca89-472a-be3e-675ee166b0ed"). InnerVolumeSpecName "kube-api-access-lg926". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:45:09.297973 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.297886    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-dshm" (OuterVolumeSpecName: "dshm") pod "e718cdac-500a-478f-8e6d-3951cdd636b0" (UID: "e718cdac-500a-478f-8e6d-3951cdd636b0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.297973 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.297886    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" (UID: "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:45:09.297973 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.297967    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de0b960c-ca89-472a-be3e-675ee166b0ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de0b960c-ca89-472a-be3e-675ee166b0ed" (UID: "de0b960c-ca89-472a-be3e-675ee166b0ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:45:09.298688 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.298666    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-kube-api-access-ldnvw" (OuterVolumeSpecName: "kube-api-access-ldnvw") pod "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" (UID: "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df"). InnerVolumeSpecName "kube-api-access-ldnvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:45:09.299226 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.299208    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-dshm" (OuterVolumeSpecName: "dshm") pod "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" (UID: "8af2f79f-738b-40d4-a4f4-eff5f2a1b4df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:45:09.396387 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396366    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de0b960c-ca89-472a-be3e-675ee166b0ed-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.396387 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396386    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e718cdac-500a-478f-8e6d-3951cdd636b0-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.396506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396395    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.396506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396402    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.396506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396411    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e718cdac-500a-478f-8e6d-3951cdd636b0-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.396506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396418    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqjhb\" (UniqueName: \"kubernetes.io/projected/e718cdac-500a-478f-8e6d-3951cdd636b0-kube-api-access-gqjhb\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.396506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396426    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de0b960c-ca89-472a-be3e-675ee166b0ed-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.396506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396434    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg926\" (UniqueName: \"kubernetes.io/projected/de0b960c-ca89-472a-be3e-675ee166b0ed-kube-api-access-lg926\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.396506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396442    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.396506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396449    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldnvw\" (UniqueName: \"kubernetes.io/projected/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-kube-api-access-ldnvw\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.396506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396459    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.396506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.396466    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:09.704276 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.704245    2562 generic.go:356] "Generic (PLEG): container finished" podID="8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" containerID="e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944" exitCode=0
Sep 16 18:45:09.704417 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.704283    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"
Sep 16 18:45:09.704417 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.704335    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb" event={"ID":"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df","Type":"ContainerDied","Data":"e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944"}
Sep 16 18:45:09.704417 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.704375    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb" event={"ID":"8af2f79f-738b-40d4-a4f4-eff5f2a1b4df","Type":"ContainerDied","Data":"51d6ab76440b4621030727793cc34ac011639a66ae307da2b1e3d2a68e218940"}
Sep 16 18:45:09.704417 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.704396    2562 scope.go:117] "RemoveContainer" containerID="e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944"
Sep 16 18:45:09.705727 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.705705    2562 generic.go:356] "Generic (PLEG): container finished" podID="de0b960c-ca89-472a-be3e-675ee166b0ed" containerID="cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0" exitCode=0
Sep 16 18:45:09.705846 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.705750    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"
Sep 16 18:45:09.705846 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.705788    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2" event={"ID":"de0b960c-ca89-472a-be3e-675ee166b0ed","Type":"ContainerDied","Data":"cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0"}
Sep 16 18:45:09.705846 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.705829    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2" event={"ID":"de0b960c-ca89-472a-be3e-675ee166b0ed","Type":"ContainerDied","Data":"1c3ae47f7a6066f2163b4e5a159b60515dd24c9aff639f40d6e71ce9351bd78a"}
Sep 16 18:45:09.707173 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.707149    2562 generic.go:356] "Generic (PLEG): container finished" podID="e718cdac-500a-478f-8e6d-3951cdd636b0" containerID="1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46" exitCode=0
Sep 16 18:45:09.707280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.707184    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k" event={"ID":"e718cdac-500a-478f-8e6d-3951cdd636b0","Type":"ContainerDied","Data":"1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46"}
Sep 16 18:45:09.707280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.707192    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"
Sep 16 18:45:09.707280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.707212    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k" event={"ID":"e718cdac-500a-478f-8e6d-3951cdd636b0","Type":"ContainerDied","Data":"10520f6c83333d769d91f3576c24e174a87700dc34aae5aaa1aabbb268accfbc"}
Sep 16 18:45:09.716382 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.716366    2562 scope.go:117] "RemoveContainer" containerID="e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944"
Sep 16 18:45:09.716677 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:45:09.716653    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944\": container with ID starting with e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944 not found: ID does not exist" containerID="e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944"
Sep 16 18:45:09.716782 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.716688    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944"} err="failed to get container status \"e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944\": rpc error: code = NotFound desc = could not find container \"e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944\": container with ID starting with e0c1bfbc6c5a0770b49eede8ce04d19192c160ce08817e1b41a91975c0617944 not found: ID does not exist"
Sep 16 18:45:09.716782 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.716709    2562 scope.go:117] "RemoveContainer" containerID="cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0"
Sep 16 18:45:09.725277 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.725261    2562 scope.go:117] "RemoveContainer" containerID="cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0"
Sep 16 18:45:09.725521 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:45:09.725504    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0\": container with ID starting with cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0 not found: ID does not exist" containerID="cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0"
Sep 16 18:45:09.725687 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.725528    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0"} err="failed to get container status \"cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0\": rpc error: code = NotFound desc = could not find container \"cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0\": container with ID starting with cf32904a72a77d848e8bf51773d4783d785df616f2542736f2651e10e93e14e0 not found: ID does not exist"
Sep 16 18:45:09.725687 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.725543    2562 scope.go:117] "RemoveContainer" containerID="1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46"
Sep 16 18:45:09.730165 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.730144    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"]
Sep 16 18:45:09.733127 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.733104    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/premium-w4-6942a4-kserve-6575bcbbcd-lqhwb"]
Sep 16 18:45:09.733639 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.733623    2562 scope.go:117] "RemoveContainer" containerID="1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46"
Sep 16 18:45:09.733888 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:45:09.733866    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46\": container with ID starting with 1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46 not found: ID does not exist" containerID="1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46"
Sep 16 18:45:09.733978 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.733895    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46"} err="failed to get container status \"1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46\": rpc error: code = NotFound desc = could not find container \"1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46\": container with ID starting with 1c4dea1861c6b9042c3c1e4f20a9572676c2a9a05abd16799de5f64efc85dc46 not found: ID does not exist"
Sep 16 18:45:09.743482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.743463    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"]
Sep 16 18:45:09.747041 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.747022    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/distinct-w4-6942a4-kserve-7f8dd7fbf7-7gg7k"]
Sep 16 18:45:09.758870 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.758851    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"]
Sep 16 18:45:09.764388 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:09.764370    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-6942a4/simulator-w4-6942a4-kserve-b68b86b55-kb6b2"]
Sep 16 18:45:10.286011 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:10.285978    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" path="/var/lib/kubelet/pods/8af2f79f-738b-40d4-a4f4-eff5f2a1b4df/volumes"
Sep 16 18:45:10.286506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:10.286485    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de0b960c-ca89-472a-be3e-675ee166b0ed" path="/var/lib/kubelet/pods/de0b960c-ca89-472a-be3e-675ee166b0ed/volumes"
Sep 16 18:45:10.286920 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:10.286902    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e718cdac-500a-478f-8e6d-3951cdd636b0" path="/var/lib/kubelet/pods/e718cdac-500a-478f-8e6d-3951cdd636b0/volumes"
Sep 16 18:45:12.273941 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:12.273913    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:12.351613 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:12.351570    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:12.586847 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:12.586762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:30.200373 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:30.200340    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:30.300010 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:30.299978    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:32.268947 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:32.268871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:34.464832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:34.464794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:37.221845 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:37.221804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:38.834772 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:38.834725    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:46.249148 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249113    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7709-4048e5"]
Sep 16 18:45:46.249524 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249509    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e718cdac-500a-478f-8e6d-3951cdd636b0" containerName="main"
Sep 16 18:45:46.249568 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249526    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e718cdac-500a-478f-8e6d-3951cdd636b0" containerName="main"
Sep 16 18:45:46.249568 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249535    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a094a321-6f0f-4719-af2c-d9dc6ca8bc24" containerName="cleanup"
Sep 16 18:45:46.249568 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249540    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a094a321-6f0f-4719-af2c-d9dc6ca8bc24" containerName="cleanup"
Sep 16 18:45:46.249568 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249551    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de0b960c-ca89-472a-be3e-675ee166b0ed" containerName="main"
Sep 16 18:45:46.249568 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249556    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="de0b960c-ca89-472a-be3e-675ee166b0ed" containerName="main"
Sep 16 18:45:46.249568 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249569    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50d8f3f3-3f6b-4440-aaa5-0c16e8104c18" containerName="payload-processing"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249575    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="50d8f3f3-3f6b-4440-aaa5-0c16e8104c18" containerName="payload-processing"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249585    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" containerName="main"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249603    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" containerName="main"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249609    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e2177b-d228-4107-8bb3-f8f0f7546d36" containerName="cleanup"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249614    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e2177b-d228-4107-8bb3-f8f0f7546d36" containerName="cleanup"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249627    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bf78544-63da-42b4-af9f-7dc6712ce018" containerName="payload-pre-processing"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249633    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bf78544-63da-42b4-af9f-7dc6712ce018" containerName="payload-pre-processing"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249707    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bf78544-63da-42b4-af9f-7dc6712ce018" containerName="payload-pre-processing"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249713    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8af2f79f-738b-40d4-a4f4-eff5f2a1b4df" containerName="main"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249721    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="62e2177b-d228-4107-8bb3-f8f0f7546d36" containerName="cleanup"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249727    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="50d8f3f3-3f6b-4440-aaa5-0c16e8104c18" containerName="payload-processing"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249732    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e718cdac-500a-478f-8e6d-3951cdd636b0" containerName="main"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249739    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="de0b960c-ca89-472a-be3e-675ee166b0ed" containerName="main"
Sep 16 18:45:46.249812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.249744    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a094a321-6f0f-4719-af2c-d9dc6ca8bc24" containerName="cleanup"
Sep 16 18:45:46.253008 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.252992    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7709-4048e5"
Sep 16 18:45:46.256139 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.256123    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xhq6g\""
Sep 16 18:45:46.261185 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.261167    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7709-4048e5"]
Sep 16 18:45:46.395059 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.395033    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgllc\" (UniqueName: \"kubernetes.io/projected/c70c2e8a-4933-4548-a9f1-390ba6ddb163-kube-api-access-xgllc\") pod \"test-curl-7709-4048e5\" (UID: \"c70c2e8a-4933-4548-a9f1-390ba6ddb163\") " pod="opendatahub/test-curl-7709-4048e5"
Sep 16 18:45:46.496040 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.496017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgllc\" (UniqueName: \"kubernetes.io/projected/c70c2e8a-4933-4548-a9f1-390ba6ddb163-kube-api-access-xgllc\") pod \"test-curl-7709-4048e5\" (UID: \"c70c2e8a-4933-4548-a9f1-390ba6ddb163\") " pod="opendatahub/test-curl-7709-4048e5"
Sep 16 18:45:46.505117 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.505064    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgllc\" (UniqueName: \"kubernetes.io/projected/c70c2e8a-4933-4548-a9f1-390ba6ddb163-kube-api-access-xgllc\") pod \"test-curl-7709-4048e5\" (UID: \"c70c2e8a-4933-4548-a9f1-390ba6ddb163\") " pod="opendatahub/test-curl-7709-4048e5"
Sep 16 18:45:46.563407 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.563381    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7709-4048e5"
Sep 16 18:45:46.831614 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.831577    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7709-4048e5"]
Sep 16 18:45:46.833695 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:45:46.833665    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc70c2e8a_4933_4548_a9f1_390ba6ddb163.slice/crio-28f85127952381eee543fac18472dfddc85f3191c10fd88a5e6e66032b643e34 WatchSource:0}: Error finding container 28f85127952381eee543fac18472dfddc85f3191c10fd88a5e6e66032b643e34: Status 404 returned error can't find the container with id 28f85127952381eee543fac18472dfddc85f3191c10fd88a5e6e66032b643e34
Sep 16 18:45:46.894983 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.894955    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7709-4048e5" event={"ID":"c70c2e8a-4933-4548-a9f1-390ba6ddb163","Type":"ContainerStarted","Data":"442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7"}
Sep 16 18:45:46.895075 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.895005    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7709-4048e5" event={"ID":"c70c2e8a-4933-4548-a9f1-390ba6ddb163","Type":"ContainerStarted","Data":"28f85127952381eee543fac18472dfddc85f3191c10fd88a5e6e66032b643e34"}
Sep 16 18:45:46.915015 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:46.914968    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7709-4048e5" podStartSLOduration=0.914953322 podStartE2EDuration="914.953322ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:46.913785203 +0000 UTC m=+1767.153573955" watchObservedRunningTime="2026-09-16 18:45:46.914953322 +0000 UTC m=+1767.154742072"
Sep 16 18:45:47.571141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:47.571101    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7709-4048e5"]
Sep 16 18:45:47.573833 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:47.573810    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7709-4048e5"]
Sep 16 18:45:47.900381 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:45:47.900313    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7709-4048e5" container="test-curl-7709-4048e5"
Sep 16 18:45:48.905746 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:48.905709    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7709-4048e5" podUID="c70c2e8a-4933-4548-a9f1-390ba6ddb163" containerName="test-curl-7709-4048e5" containerID="cri-o://442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7" gracePeriod=2
Sep 16 18:45:48.920580 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:48.920556    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:49.947280 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:49.947244    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:51.090203 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:51.090180    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7709-4048e5"
Sep 16 18:45:51.233437 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:51.233410    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgllc\" (UniqueName: \"kubernetes.io/projected/c70c2e8a-4933-4548-a9f1-390ba6ddb163-kube-api-access-xgllc\") pod \"c70c2e8a-4933-4548-a9f1-390ba6ddb163\" (UID: \"c70c2e8a-4933-4548-a9f1-390ba6ddb163\") "
Sep 16 18:45:51.235438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:51.235413    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c70c2e8a-4933-4548-a9f1-390ba6ddb163-kube-api-access-xgllc" (OuterVolumeSpecName: "kube-api-access-xgllc") pod "c70c2e8a-4933-4548-a9f1-390ba6ddb163" (UID: "c70c2e8a-4933-4548-a9f1-390ba6ddb163"). InnerVolumeSpecName "kube-api-access-xgllc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:45:51.334686 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:51.334667    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgllc\" (UniqueName: \"kubernetes.io/projected/c70c2e8a-4933-4548-a9f1-390ba6ddb163-kube-api-access-xgllc\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:45:51.928040 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:51.928011    2562 generic.go:356] "Generic (PLEG): container finished" podID="c70c2e8a-4933-4548-a9f1-390ba6ddb163" containerID="442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7" exitCode=137
Sep 16 18:45:51.928145 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:51.928051    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7709-4048e5"
Sep 16 18:45:51.928145 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:51.928108    2562 scope.go:117] "RemoveContainer" containerID="442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7"
Sep 16 18:45:51.936794 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:51.936778    2562 scope.go:117] "RemoveContainer" containerID="442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7"
Sep 16 18:45:51.937076 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:45:51.937059    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7\": container with ID starting with 442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7 not found: ID does not exist" containerID="442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7"
Sep 16 18:45:51.937141 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:51.937087    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7"} err="failed to get container status \"442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7\": rpc error: code = NotFound desc = could not find container \"442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7\": container with ID starting with 442eba3431b08e9da3225e42838d9866b04eb77e36fcd41ced6bf85b4f4a96c7 not found: ID does not exist"
Sep 16 18:45:51.937746 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:45:51.937721    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7709-4048e5\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="c70c2e8a-4933-4548-a9f1-390ba6ddb163" pod="opendatahub/test-curl-7709-4048e5"
Sep 16 18:45:52.285633 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:52.285540    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c70c2e8a-4933-4548-a9f1-390ba6ddb163" path="/var/lib/kubelet/pods/c70c2e8a-4933-4548-a9f1-390ba6ddb163/volumes"
Sep 16 18:45:59.344501 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:45:59.344471    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:19.393104 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:46:19.393061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:20.618555 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:46:20.618525    2562 scope.go:117] "RemoveContainer" containerID="7ca15c7a022e07400b1ece3016b3869d0ae8f51362d3aa264b7d82dda21a819c"
Sep 16 18:46:57.430914 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:46:57.430874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:58.478103 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:46:58.478075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:00.583965 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:47:00.583934    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:02.127800 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:47:02.127768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:39.450541 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:47:39.450505    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:40.828608 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:47:40.828560    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:09.500231 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:09.500196    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:22.516008 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:22.515961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:23.333899 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:23.333865    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:23.479877 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:23.479850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:23.760126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:23.760099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:24.098510 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:24.098430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:24.513512 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:24.513479    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:26.479240 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:26.479205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:27.151780 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:27.151746    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:27.253269 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:27.253238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:27.689301 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:27.689274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:28.902511 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:28.902477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:30.387296 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:30.387263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:30.484068 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:30.484041    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:30.695132 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:30.695096    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:31.116538 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:31.116456    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:31.245216 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:31.245179    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:54.190669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:54.190635    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:55.213338 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:55.213300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:55.972322 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:55.972287    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:56.681485 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:56.681444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:57.194372 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:57.194341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:57.847975 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:57.847949    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:58.186344 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:58.186315    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:58.290020 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:48:58.289973    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:00.903673 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:00.903635    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:01.571208 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:01.571175    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:01.667285 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:01.667252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:01.902859 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:01.902766    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:02.663643 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:02.663611    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:03.594205 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:03.594173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:04.368414 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:04.368362    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:04.468390 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:04.468365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:04.732206 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:04.732178    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:04.915403 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:04.915375    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:05.015415 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:05.015349    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:05.259514 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:05.259478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.297973 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:06.297937    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:20.658742 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:20.658716    2562 scope.go:117] "RemoveContainer" containerID="8440d40a8723448e3c7a3c441384025f0b468ff8f3547e2ad2d7c352fa149adf"
Sep 16 18:49:26.529196 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:26.529163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:26.846979 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:26.846900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:27.888105 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:27.888070    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:29.236622 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:29.236573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:30.567269 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:30.567237    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:31.696762 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:31.696728    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pjwhq_1963517c-bd16-44cd-9994-8a432db84f92/payload-processing/0.log"
Sep 16 18:49:31.799101 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:31.799074    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:32.240258 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:32.240225    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:46.596131 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.596102    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"]
Sep 16 18:49:46.596555 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.596490    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c70c2e8a-4933-4548-a9f1-390ba6ddb163" containerName="test-curl-7709-4048e5"
Sep 16 18:49:46.596555 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.596502    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c70c2e8a-4933-4548-a9f1-390ba6ddb163" containerName="test-curl-7709-4048e5"
Sep 16 18:49:46.596658 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.596565    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c70c2e8a-4933-4548-a9f1-390ba6ddb163" containerName="test-curl-7709-4048e5"
Sep 16 18:49:46.599813 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.599796    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.602303 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.602281    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6817d3\"/\"openshift-service-ca.crt\""
Sep 16 18:49:46.603195 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.603175    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6817d3\"/\"kube-root-ca.crt\""
Sep 16 18:49:46.603267 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.603256    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6817d3\"/\"default-dockercfg-pp4pk\""
Sep 16 18:49:46.603311 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.603279    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6817d3\"/\"ipp-route-6817d3-kserve-self-signed-certs\""
Sep 16 18:49:46.610544 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.610519    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-model-cache\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.610700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.610562    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b543323-d070-43d7-aad8-41b6f1248a9c-tls-certs\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.610700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.610625    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"]
Sep 16 18:49:46.610700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.610653    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcdmw\" (UniqueName: \"kubernetes.io/projected/6b543323-d070-43d7-aad8-41b6f1248a9c-kube-api-access-mcdmw\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.610700 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.610672    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-dshm\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.610894 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.610706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-home\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.610894 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.610737    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-tmp-dir\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.628033 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.628008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:46.711519 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.711493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcdmw\" (UniqueName: \"kubernetes.io/projected/6b543323-d070-43d7-aad8-41b6f1248a9c-kube-api-access-mcdmw\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.711659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.711527    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-dshm\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.711659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.711571    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-home\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.711659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.711616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-tmp-dir\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.711833 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.711686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-model-cache\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.711833 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.711739    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b543323-d070-43d7-aad8-41b6f1248a9c-tls-certs\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.712024 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.712002    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-home\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.712113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.712051    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-tmp-dir\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.712113 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.712076    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-model-cache\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.713965 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.713944    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-dshm\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.714471 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.714454    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b543323-d070-43d7-aad8-41b6f1248a9c-tls-certs\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.725892 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.725872    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcdmw\" (UniqueName: \"kubernetes.io/projected/6b543323-d070-43d7-aad8-41b6f1248a9c-kube-api-access-mcdmw\") pod \"ipp-route-6817d3-kserve-578f7bd667-d67pk\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") " pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:46.909610 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:46.909545    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:47.182131 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:47.182107    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"]
Sep 16 18:49:47.183902 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:49:47.183876    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b543323_d070_43d7_aad8_41b6f1248a9c.slice/crio-64c7fed0ec5e5d4fd5d108c2e88ffb713710196b129b0c43b54cd30e6b8fe498 WatchSource:0}: Error finding container 64c7fed0ec5e5d4fd5d108c2e88ffb713710196b129b0c43b54cd30e6b8fe498: Status 404 returned error can't find the container with id 64c7fed0ec5e5d4fd5d108c2e88ffb713710196b129b0c43b54cd30e6b8fe498
Sep 16 18:49:48.102854 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:48.102825    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk" event={"ID":"6b543323-d070-43d7-aad8-41b6f1248a9c","Type":"ContainerStarted","Data":"30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9"}
Sep 16 18:49:48.103216 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:48.102861    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk" event={"ID":"6b543323-d070-43d7-aad8-41b6f1248a9c","Type":"ContainerStarted","Data":"64c7fed0ec5e5d4fd5d108c2e88ffb713710196b129b0c43b54cd30e6b8fe498"}
Sep 16 18:49:48.123460 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:48.123413    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk" podStartSLOduration=2.123398826 podStartE2EDuration="2.123398826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:48.121309335 +0000 UTC m=+2008.361098088" watchObservedRunningTime="2026-09-16 18:49:48.123398826 +0000 UTC m=+2008.363187577"
Sep 16 18:49:56.909932 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:56.909890    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:56.910319 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:56.909966    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:56.922059 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:56.922039    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:49:57.157434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:49:57.157411    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:50:07.601834 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:07.601803    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:14.313103 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:14.313069    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:18.109251 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:18.109220    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pjwhq_1963517c-bd16-44cd-9994-8a432db84f92/payload-processing/0.log"
Sep 16 18:50:18.501337 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:18.501302    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:18.794417 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:18.794332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:18.904878 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:18.904846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:20.682246 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:20.682221    2562 scope.go:117] "RemoveContainer" containerID="e58402f757c5a74143821b95e8da425f71959f5ee0f555592e556459c9e2dfe3"
Sep 16 18:50:20.692857 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:20.692836    2562 scope.go:117] "RemoveContainer" containerID="eaad01e09b71cf6947c13f877e9d9322e554e09a166e8c49b79d01faaca74a8b"
Sep 16 18:50:20.700553 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:20.700526    2562 scope.go:117] "RemoveContainer" containerID="f6894f03e1cb842b4767e6efc3a39f0a7369686a278ee92522477160f6495bbc"
Sep 16 18:50:21.165475 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:21.165443    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:21.882716 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:21.882681    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:21.967006 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:21.966966    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:22.849445 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:22.849420    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:23.669354 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:23.669323    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:24.438245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:24.438218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:25.286899 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.286868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:25.383277 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.383254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:25.718004 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.717973    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:25.806502 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.806475    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"]
Sep 16 18:50:25.809014 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.808979    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:25.810011 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.809991    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:50:25.812340 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.812322    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-ebbe9a\""
Sep 16 18:50:25.822444 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.822423    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"]
Sep 16 18:50:25.835939 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.835919    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"]
Sep 16 18:50:25.839478 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.839453    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:50:25.842048 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.842030    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-ebbe9a-dockercfg-vwstb\""
Sep 16 18:50:25.850413 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.850395    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"]
Sep 16 18:50:25.910167 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.910146    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73301d7e-2c19-4a1d-9304-f64b99fb48b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx\" (UID: \"73301d7e-2c19-4a1d-9304-f64b99fb48b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:50:25.910261 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.910176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f8ef82-aef7-44d6-9b41-035ff900565e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp\" (UID: \"59f8ef82-aef7-44d6-9b41-035ff900565e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:50:25.910261 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.910201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzjvx\" (UniqueName: \"kubernetes.io/projected/73301d7e-2c19-4a1d-9304-f64b99fb48b8-kube-api-access-zzjvx\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx\" (UID: \"73301d7e-2c19-4a1d-9304-f64b99fb48b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:50:25.910339 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:25.910260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mgcr\" (UniqueName: \"kubernetes.io/projected/59f8ef82-aef7-44d6-9b41-035ff900565e-kube-api-access-5mgcr\") pod \"payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp\" (UID: \"59f8ef82-aef7-44d6-9b41-035ff900565e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:50:26.011347 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.011297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73301d7e-2c19-4a1d-9304-f64b99fb48b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx\" (UID: \"73301d7e-2c19-4a1d-9304-f64b99fb48b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:50:26.011347 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.011326    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f8ef82-aef7-44d6-9b41-035ff900565e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp\" (UID: \"59f8ef82-aef7-44d6-9b41-035ff900565e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:50:26.011347 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.011347    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzjvx\" (UniqueName: \"kubernetes.io/projected/73301d7e-2c19-4a1d-9304-f64b99fb48b8-kube-api-access-zzjvx\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx\" (UID: \"73301d7e-2c19-4a1d-9304-f64b99fb48b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:50:26.011558 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.011377    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mgcr\" (UniqueName: \"kubernetes.io/projected/59f8ef82-aef7-44d6-9b41-035ff900565e-kube-api-access-5mgcr\") pod \"payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp\" (UID: \"59f8ef82-aef7-44d6-9b41-035ff900565e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:50:26.012023 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.012006    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73301d7e-2c19-4a1d-9304-f64b99fb48b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx\" (UID: \"73301d7e-2c19-4a1d-9304-f64b99fb48b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:50:26.012064 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.012044    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f8ef82-aef7-44d6-9b41-035ff900565e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp\" (UID: \"59f8ef82-aef7-44d6-9b41-035ff900565e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:50:26.019792 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.019767    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzjvx\" (UniqueName: \"kubernetes.io/projected/73301d7e-2c19-4a1d-9304-f64b99fb48b8-kube-api-access-zzjvx\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx\" (UID: \"73301d7e-2c19-4a1d-9304-f64b99fb48b8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:50:26.020015 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.020002    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mgcr\" (UniqueName: \"kubernetes.io/projected/59f8ef82-aef7-44d6-9b41-035ff900565e-kube-api-access-5mgcr\") pod \"payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp\" (UID: \"59f8ef82-aef7-44d6-9b41-035ff900565e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:50:26.120045 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.120025    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:50:26.150252 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.150232    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:50:26.396232 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.396208    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"]
Sep 16 18:50:26.397074 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:50:26.397054    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73301d7e_2c19_4a1d_9304_f64b99fb48b8.slice/crio-a1912c7fe24141d722551a2ae643128a0e0196b9d95bcf7a326dde2809778eee WatchSource:0}: Error finding container a1912c7fe24141d722551a2ae643128a0e0196b9d95bcf7a326dde2809778eee: Status 404 returned error can't find the container with id a1912c7fe24141d722551a2ae643128a0e0196b9d95bcf7a326dde2809778eee
Sep 16 18:50:26.430840 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:26.430815    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"]
Sep 16 18:50:26.432707 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:50:26.432684    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f8ef82_aef7_44d6_9b41_035ff900565e.slice/crio-8fd0328bac0e4409a93dc1d6d1c1d06c31df9d46c95e58498c93d249d13bb3f6 WatchSource:0}: Error finding container 8fd0328bac0e4409a93dc1d6d1c1d06c31df9d46c95e58498c93d249d13bb3f6: Status 404 returned error can't find the container with id 8fd0328bac0e4409a93dc1d6d1c1d06c31df9d46c95e58498c93d249d13bb3f6
Sep 16 18:50:27.287726 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:27.287689    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp" event={"ID":"59f8ef82-aef7-44d6-9b41-035ff900565e","Type":"ContainerStarted","Data":"0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7"}
Sep 16 18:50:27.287912 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:27.287750    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp" event={"ID":"59f8ef82-aef7-44d6-9b41-035ff900565e","Type":"ContainerStarted","Data":"8fd0328bac0e4409a93dc1d6d1c1d06c31df9d46c95e58498c93d249d13bb3f6"}
Sep 16 18:50:27.287912 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:27.287782    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:50:27.289158 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:27.289134    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx" event={"ID":"73301d7e-2c19-4a1d-9304-f64b99fb48b8","Type":"ContainerStarted","Data":"aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4"}
Sep 16 18:50:27.289274 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:27.289177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx" event={"ID":"73301d7e-2c19-4a1d-9304-f64b99fb48b8","Type":"ContainerStarted","Data":"a1912c7fe24141d722551a2ae643128a0e0196b9d95bcf7a326dde2809778eee"}
Sep 16 18:50:27.289274 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:27.289266    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:50:27.307571 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:27.307528    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp" podStartSLOduration=2.307516676 podStartE2EDuration="2.307516676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:50:27.304930653 +0000 UTC m=+2047.544719405" watchObservedRunningTime="2026-09-16 18:50:27.307516676 +0000 UTC m=+2047.547305428"
Sep 16 18:50:27.327272 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:27.327225    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx" podStartSLOduration=2.327213394 podStartE2EDuration="2.327213394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:50:27.325148688 +0000 UTC m=+2047.564937431" watchObservedRunningTime="2026-09-16 18:50:27.327213394 +0000 UTC m=+2047.567002190"
Sep 16 18:50:38.295662 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:38.295636    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:50:38.296080 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:38.295695    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:50:47.240826 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:47.240749    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:47.998665 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:47.998633    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:48.662982 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:48.662934    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:49.524720 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:49.524681    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:50.300551 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:50.300520    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:50.989919 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:50.989893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:52.021739 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:52.021708    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:52.705795 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:52.705767    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"]
Sep 16 18:50:52.710450 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:52.710432    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"
Sep 16 18:50:52.712946 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:52.712928    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:50:52.713036 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:52.712989    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:50:52.718251 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:52.718227    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"]
Sep 16 18:50:52.817304 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:52.817282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4952889b-2875-4c35-a6a2-853b37d5327a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x\" (UID: \"4952889b-2875-4c35-a6a2-853b37d5327a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"
Sep 16 18:50:52.918110 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:52.918084    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4952889b-2875-4c35-a6a2-853b37d5327a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x\" (UID: \"4952889b-2875-4c35-a6a2-853b37d5327a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"
Sep 16 18:50:52.918713 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:52.918693    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4952889b-2875-4c35-a6a2-853b37d5327a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x\" (UID: \"4952889b-2875-4c35-a6a2-853b37d5327a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"
Sep 16 18:50:53.021901 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:53.021847    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"
Sep 16 18:50:53.294040 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:53.294015    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"]
Sep 16 18:50:53.295706 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:50:53.295683    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4952889b_2875_4c35_a6a2_853b37d5327a.slice/crio-e1ae1a70f4bab0382b550baed163b0e8a305021a99733cd0956536ff6eefa91c WatchSource:0}: Error finding container e1ae1a70f4bab0382b550baed163b0e8a305021a99733cd0956536ff6eefa91c: Status 404 returned error can't find the container with id e1ae1a70f4bab0382b550baed163b0e8a305021a99733cd0956536ff6eefa91c
Sep 16 18:50:53.422142 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:53.422114    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x" event={"ID":"4952889b-2875-4c35-a6a2-853b37d5327a","Type":"ContainerStarted","Data":"e1ae1a70f4bab0382b550baed163b0e8a305021a99733cd0956536ff6eefa91c"}
Sep 16 18:50:53.611414 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:53.611386    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:54.396514 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:54.396436    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:54.427821 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:54.427796    2562 generic.go:356] "Generic (PLEG): container finished" podID="4952889b-2875-4c35-a6a2-853b37d5327a" containerID="c9d7142fc27eaf11d277eff25ff352c77b0a5ff3fbade998c47dc223e7bc700f" exitCode=0
Sep 16 18:50:54.427939 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:54.427831    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x" event={"ID":"4952889b-2875-4c35-a6a2-853b37d5327a","Type":"ContainerDied","Data":"c9d7142fc27eaf11d277eff25ff352c77b0a5ff3fbade998c47dc223e7bc700f"}
Sep 16 18:50:55.229564 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:55.229535    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:55.563549 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:55.563527    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"
Sep 16 18:50:55.642076 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:55.642049    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4952889b-2875-4c35-a6a2-853b37d5327a-maas-api-service-ca\") pod \"4952889b-2875-4c35-a6a2-853b37d5327a\" (UID: \"4952889b-2875-4c35-a6a2-853b37d5327a\") "
Sep 16 18:50:55.642363 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:55.642341    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4952889b-2875-4c35-a6a2-853b37d5327a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4952889b-2875-4c35-a6a2-853b37d5327a" (UID: "4952889b-2875-4c35-a6a2-853b37d5327a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:50:55.743267 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:55.743207    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4952889b-2875-4c35-a6a2-853b37d5327a-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:50:55.942832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:55.942802    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:56.438088 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:56.438063    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"
Sep 16 18:50:56.438198 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:56.438108    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x" event={"ID":"4952889b-2875-4c35-a6a2-853b37d5327a","Type":"ContainerDied","Data":"e1ae1a70f4bab0382b550baed163b0e8a305021a99733cd0956536ff6eefa91c"}
Sep 16 18:50:56.438198 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:56.438145    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1ae1a70f4bab0382b550baed163b0e8a305021a99733cd0956536ff6eefa91c"
Sep 16 18:50:56.613506 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:56.613439    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:57.316133 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:50:57.316107    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:03.673861 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.673833    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:03.696076 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.696044    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"]
Sep 16 18:51:03.696309 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.696283    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp" podUID="59f8ef82-aef7-44d6-9b41-035ff900565e" containerName="payload-processing" containerID="cri-o://0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7" gracePeriod=30
Sep 16 18:51:03.704304 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.704279    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"]
Sep 16 18:51:03.704984 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.704941    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx" podUID="73301d7e-2c19-4a1d-9304-f64b99fb48b8" containerName="payload-pre-processing" containerID="cri-o://aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4" gracePeriod=30
Sep 16 18:51:03.785918 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.785888    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:03.866549 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.866516    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"]
Sep 16 18:51:03.868478 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.868444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:03.868865 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.868841    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ebbe9a-cs88x"]
Sep 16 18:51:03.938006 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.937933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:03.982219 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.982199    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:51:03.985044 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.985017    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:51:03.996505 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.996485    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73301d7e-2c19-4a1d-9304-f64b99fb48b8-plugins-config-volume\") pod \"73301d7e-2c19-4a1d-9304-f64b99fb48b8\" (UID: \"73301d7e-2c19-4a1d-9304-f64b99fb48b8\") "
Sep 16 18:51:03.996599 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.996528    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzjvx\" (UniqueName: \"kubernetes.io/projected/73301d7e-2c19-4a1d-9304-f64b99fb48b8-kube-api-access-zzjvx\") pod \"73301d7e-2c19-4a1d-9304-f64b99fb48b8\" (UID: \"73301d7e-2c19-4a1d-9304-f64b99fb48b8\") "
Sep 16 18:51:03.996599 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.996574    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mgcr\" (UniqueName: \"kubernetes.io/projected/59f8ef82-aef7-44d6-9b41-035ff900565e-kube-api-access-5mgcr\") pod \"59f8ef82-aef7-44d6-9b41-035ff900565e\" (UID: \"59f8ef82-aef7-44d6-9b41-035ff900565e\") "
Sep 16 18:51:03.996698 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.996625    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f8ef82-aef7-44d6-9b41-035ff900565e-plugins-config-volume\") pod \"59f8ef82-aef7-44d6-9b41-035ff900565e\" (UID: \"59f8ef82-aef7-44d6-9b41-035ff900565e\") "
Sep 16 18:51:03.996927 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.996892    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73301d7e-2c19-4a1d-9304-f64b99fb48b8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "73301d7e-2c19-4a1d-9304-f64b99fb48b8" (UID: "73301d7e-2c19-4a1d-9304-f64b99fb48b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:51:03.997004 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.996962    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73301d7e-2c19-4a1d-9304-f64b99fb48b8-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:51:03.997004 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.996979    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59f8ef82-aef7-44d6-9b41-035ff900565e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "59f8ef82-aef7-44d6-9b41-035ff900565e" (UID: "59f8ef82-aef7-44d6-9b41-035ff900565e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:51:03.998691 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.998662    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73301d7e-2c19-4a1d-9304-f64b99fb48b8-kube-api-access-zzjvx" (OuterVolumeSpecName: "kube-api-access-zzjvx") pod "73301d7e-2c19-4a1d-9304-f64b99fb48b8" (UID: "73301d7e-2c19-4a1d-9304-f64b99fb48b8"). InnerVolumeSpecName "kube-api-access-zzjvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:51:03.998887 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:03.998744    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59f8ef82-aef7-44d6-9b41-035ff900565e-kube-api-access-5mgcr" (OuterVolumeSpecName: "kube-api-access-5mgcr") pod "59f8ef82-aef7-44d6-9b41-035ff900565e" (UID: "59f8ef82-aef7-44d6-9b41-035ff900565e"). InnerVolumeSpecName "kube-api-access-5mgcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:51:04.097393 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.097363    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f8ef82-aef7-44d6-9b41-035ff900565e-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:51:04.097393 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.097388    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzjvx\" (UniqueName: \"kubernetes.io/projected/73301d7e-2c19-4a1d-9304-f64b99fb48b8-kube-api-access-zzjvx\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:51:04.097393 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.097397    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mgcr\" (UniqueName: \"kubernetes.io/projected/59f8ef82-aef7-44d6-9b41-035ff900565e-kube-api-access-5mgcr\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:51:04.262254 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.262185    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:04.286230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.286204    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4952889b-2875-4c35-a6a2-853b37d5327a" path="/var/lib/kubelet/pods/4952889b-2875-4c35-a6a2-853b37d5327a/volumes"
Sep 16 18:51:04.362060 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.362025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:04.479735 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.479710    2562 generic.go:356] "Generic (PLEG): container finished" podID="59f8ef82-aef7-44d6-9b41-035ff900565e" containerID="0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7" exitCode=0
Sep 16 18:51:04.479829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.479758    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"
Sep 16 18:51:04.479829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.479767    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp" event={"ID":"59f8ef82-aef7-44d6-9b41-035ff900565e","Type":"ContainerDied","Data":"0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7"}
Sep 16 18:51:04.479829 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.479815    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp" event={"ID":"59f8ef82-aef7-44d6-9b41-035ff900565e","Type":"ContainerDied","Data":"8fd0328bac0e4409a93dc1d6d1c1d06c31df9d46c95e58498c93d249d13bb3f6"}
Sep 16 18:51:04.479997 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.479837    2562 scope.go:117] "RemoveContainer" containerID="0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7"
Sep 16 18:51:04.481239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.481218    2562 generic.go:356] "Generic (PLEG): container finished" podID="73301d7e-2c19-4a1d-9304-f64b99fb48b8" containerID="aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4" exitCode=0
Sep 16 18:51:04.481339 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.481279    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"
Sep 16 18:51:04.481399 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.481283    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx" event={"ID":"73301d7e-2c19-4a1d-9304-f64b99fb48b8","Type":"ContainerDied","Data":"aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4"}
Sep 16 18:51:04.481465 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.481446    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx" event={"ID":"73301d7e-2c19-4a1d-9304-f64b99fb48b8","Type":"ContainerDied","Data":"a1912c7fe24141d722551a2ae643128a0e0196b9d95bcf7a326dde2809778eee"}
Sep 16 18:51:04.489112 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.489096    2562 scope.go:117] "RemoveContainer" containerID="0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7"
Sep 16 18:51:04.489386 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:51:04.489369    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7\": container with ID starting with 0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7 not found: ID does not exist" containerID="0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7"
Sep 16 18:51:04.489430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.489395    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7"} err="failed to get container status \"0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7\": rpc error: code = NotFound desc = could not find container \"0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7\": container with ID starting with 0e45704018fbe40c88d725d9a7d6a28d44106a7b09525ad1732f5863ffb584a7 not found: ID does not exist"
Sep 16 18:51:04.489430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.489412    2562 scope.go:117] "RemoveContainer" containerID="aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4"
Sep 16 18:51:04.496756 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.496735    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"]
Sep 16 18:51:04.498500 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.498449    2562 scope.go:117] "RemoveContainer" containerID="aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4"
Sep 16 18:51:04.498753 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:51:04.498732    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4\": container with ID starting with aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4 not found: ID does not exist" containerID="aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4"
Sep 16 18:51:04.498821 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.498763    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4"} err="failed to get container status \"aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4\": rpc error: code = NotFound desc = could not find container \"aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4\": container with ID starting with aaa49e7d1b6fdd4c235c8a3ef6bc77bc24f52836f2ce6b5c30f55456cb824ab4 not found: ID does not exist"
Sep 16 18:51:04.500304 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.500287    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebbe9a-ff6894759-6jjjp"]
Sep 16 18:51:04.510850 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.510830    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"]
Sep 16 18:51:04.514247 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.514200    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebbe9a-87b466f58-wdvlx"]
Sep 16 18:51:04.995143 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:04.995122    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:05.749074 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:05.749042    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:06.285665 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:06.285641    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59f8ef82-aef7-44d6-9b41-035ff900565e" path="/var/lib/kubelet/pods/59f8ef82-aef7-44d6-9b41-035ff900565e/volumes"
Sep 16 18:51:06.286030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:06.285965    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73301d7e-2c19-4a1d-9304-f64b99fb48b8" path="/var/lib/kubelet/pods/73301d7e-2c19-4a1d-9304-f64b99fb48b8/volumes"
Sep 16 18:51:06.512267 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:06.512238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:07.228522 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:07.228489    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:10.304058 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:10.304021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:10.406186 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:10.406155    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:10.507547 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:10.507517    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:10.694899 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:10.694875    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:14.146005 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.145971    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"]
Sep 16 18:51:14.146380 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.146362    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4952889b-2875-4c35-a6a2-853b37d5327a" containerName="revoke-keys"
Sep 16 18:51:14.146380 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.146381    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4952889b-2875-4c35-a6a2-853b37d5327a" containerName="revoke-keys"
Sep 16 18:51:14.146481 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.146394    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73301d7e-2c19-4a1d-9304-f64b99fb48b8" containerName="payload-pre-processing"
Sep 16 18:51:14.146481 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.146399    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="73301d7e-2c19-4a1d-9304-f64b99fb48b8" containerName="payload-pre-processing"
Sep 16 18:51:14.146481 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.146412    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59f8ef82-aef7-44d6-9b41-035ff900565e" containerName="payload-processing"
Sep 16 18:51:14.146481 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.146418    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="59f8ef82-aef7-44d6-9b41-035ff900565e" containerName="payload-processing"
Sep 16 18:51:14.146481 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.146480    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4952889b-2875-4c35-a6a2-853b37d5327a" containerName="revoke-keys"
Sep 16 18:51:14.146659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.146488    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="59f8ef82-aef7-44d6-9b41-035ff900565e" containerName="payload-processing"
Sep 16 18:51:14.146659 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.146495    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="73301d7e-2c19-4a1d-9304-f64b99fb48b8" containerName="payload-pre-processing"
Sep 16 18:51:14.149738 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.149723    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"
Sep 16 18:51:14.152081 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.152058    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:51:14.152230 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.152158    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:51:14.157819 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.157381    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"]
Sep 16 18:51:14.176758 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.176731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13b95d3-ee2b-4e49-805c-01eb4a624109-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs\" (UID: \"f13b95d3-ee2b-4e49-805c-01eb4a624109\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"
Sep 16 18:51:14.277773 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.277752    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13b95d3-ee2b-4e49-805c-01eb4a624109-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs\" (UID: \"f13b95d3-ee2b-4e49-805c-01eb4a624109\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"
Sep 16 18:51:14.278271 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.278255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13b95d3-ee2b-4e49-805c-01eb4a624109-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs\" (UID: \"f13b95d3-ee2b-4e49-805c-01eb4a624109\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"
Sep 16 18:51:14.320801 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.320776    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:14.460541 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.460518    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"
Sep 16 18:51:14.730387 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:14.730361    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"]
Sep 16 18:51:14.732481 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:51:14.732446    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13b95d3_ee2b_4e49_805c_01eb4a624109.slice/crio-e5d3f0e552f93698d1b13b6d47a068c814e70de9b9afaaa8c136b6ad4047b66a WatchSource:0}: Error finding container e5d3f0e552f93698d1b13b6d47a068c814e70de9b9afaaa8c136b6ad4047b66a: Status 404 returned error can't find the container with id e5d3f0e552f93698d1b13b6d47a068c814e70de9b9afaaa8c136b6ad4047b66a
Sep 16 18:51:15.027075 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:15.027020    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:15.539882 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:15.539851    2562 generic.go:356] "Generic (PLEG): container finished" podID="f13b95d3-ee2b-4e49-805c-01eb4a624109" containerID="4fc2c23e53ad45a0a74c5f7bb3ebd9f5e6ba0b25b0140444c25ae34c37f18947" exitCode=0
Sep 16 18:51:15.540167 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:15.539931    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs" event={"ID":"f13b95d3-ee2b-4e49-805c-01eb4a624109","Type":"ContainerDied","Data":"4fc2c23e53ad45a0a74c5f7bb3ebd9f5e6ba0b25b0140444c25ae34c37f18947"}
Sep 16 18:51:15.540167 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:15.539979    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs" event={"ID":"f13b95d3-ee2b-4e49-805c-01eb4a624109","Type":"ContainerStarted","Data":"e5d3f0e552f93698d1b13b6d47a068c814e70de9b9afaaa8c136b6ad4047b66a"}
Sep 16 18:51:15.826287 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:15.826219    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:16.522032 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:16.521999    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:16.675971 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:16.675947    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"
Sep 16 18:51:16.698581 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:16.698560    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13b95d3-ee2b-4e49-805c-01eb4a624109-maas-api-service-ca\") pod \"f13b95d3-ee2b-4e49-805c-01eb4a624109\" (UID: \"f13b95d3-ee2b-4e49-805c-01eb4a624109\") "
Sep 16 18:51:16.698944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:16.698918    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13b95d3-ee2b-4e49-805c-01eb4a624109-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f13b95d3-ee2b-4e49-805c-01eb4a624109" (UID: "f13b95d3-ee2b-4e49-805c-01eb4a624109"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:51:16.799683 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:16.799631    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13b95d3-ee2b-4e49-805c-01eb4a624109-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:51:17.236669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:17.236579    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:17.551935 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:17.551868    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"
Sep 16 18:51:17.551935 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:17.551883    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs" event={"ID":"f13b95d3-ee2b-4e49-805c-01eb4a624109","Type":"ContainerDied","Data":"e5d3f0e552f93698d1b13b6d47a068c814e70de9b9afaaa8c136b6ad4047b66a"}
Sep 16 18:51:17.551935 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:17.551921    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5d3f0e552f93698d1b13b6d47a068c814e70de9b9afaaa8c136b6ad4047b66a"
Sep 16 18:51:17.900430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:17.900359    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:20.727786 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:20.727762    2562 scope.go:117] "RemoveContainer" containerID="cf1d87ef5a513ad2f3f42751f95b414bf9e3683284c0f388514ee08b657e7de3"
Sep 16 18:51:20.736551 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:20.736535    2562 scope.go:117] "RemoveContainer" containerID="970e45d7ea802f4fe89911e15b57fb58e5bfa0cabf516bd6e713c76350c1fdc0"
Sep 16 18:51:20.745040 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:20.745022    2562 scope.go:117] "RemoveContainer" containerID="55fd156e83a0a82b34566d99b01d2bb741ea4fb04ba6c85e41641efd0b966607"
Sep 16 18:51:20.753237 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:20.753219    2562 scope.go:117] "RemoveContainer" containerID="0788812be8c507bd06b28f0e37e34de9b52ab2da0bc0695ef24569e811c42a5d"
Sep 16 18:51:25.693910 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:25.693878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:25.764111 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:25.764084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:29.963536 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:29.963494    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:30.043390 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:30.043355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:30.151911 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:30.151881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:30.244641 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:30.244548    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:30.245701 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:30.245684    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"]
Sep 16 18:51:30.252003 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:30.251982    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6817d3-xhmbs"]
Sep 16 18:51:30.285578 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:30.285552    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13b95d3-ee2b-4e49-805c-01eb4a624109" path="/var/lib/kubelet/pods/f13b95d3-ee2b-4e49-805c-01eb4a624109/volumes"
Sep 16 18:51:30.306263 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:30.306235    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:30.605877 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:30.605806    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:30.704405 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:30.704376    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:31.421532 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:31.421502    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:32.279001 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:32.278961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:33.071132 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:33.071100    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:35.580548 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:35.580518    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"]
Sep 16 18:51:35.677070 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:35.677040    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:50.598663 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.598609    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk" podUID="6b543323-d070-43d7-aad8-41b6f1248a9c" containerName="main" containerID="cri-o://30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9" gracePeriod=45
Sep 16 18:51:50.851026 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.850965    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:51:50.956124 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.956092    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b543323-d070-43d7-aad8-41b6f1248a9c-tls-certs\") pod \"6b543323-d070-43d7-aad8-41b6f1248a9c\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") "
Sep 16 18:51:50.956264 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.956139    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-model-cache\") pod \"6b543323-d070-43d7-aad8-41b6f1248a9c\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") "
Sep 16 18:51:50.956264 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.956157    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-tmp-dir\") pod \"6b543323-d070-43d7-aad8-41b6f1248a9c\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") "
Sep 16 18:51:50.956264 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.956201    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-dshm\") pod \"6b543323-d070-43d7-aad8-41b6f1248a9c\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") "
Sep 16 18:51:50.956264 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.956228    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-home\") pod \"6b543323-d070-43d7-aad8-41b6f1248a9c\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") "
Sep 16 18:51:50.956437 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.956275    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcdmw\" (UniqueName: \"kubernetes.io/projected/6b543323-d070-43d7-aad8-41b6f1248a9c-kube-api-access-mcdmw\") pod \"6b543323-d070-43d7-aad8-41b6f1248a9c\" (UID: \"6b543323-d070-43d7-aad8-41b6f1248a9c\") "
Sep 16 18:51:50.956437 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.956389    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b543323-d070-43d7-aad8-41b6f1248a9c" (UID: "6b543323-d070-43d7-aad8-41b6f1248a9c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:50.956550 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.956476    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-home" (OuterVolumeSpecName: "home") pod "6b543323-d070-43d7-aad8-41b6f1248a9c" (UID: "6b543323-d070-43d7-aad8-41b6f1248a9c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:50.956550 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.956481    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b543323-d070-43d7-aad8-41b6f1248a9c" (UID: "6b543323-d070-43d7-aad8-41b6f1248a9c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:50.956680 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.956553    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:51:50.958386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.958365    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-dshm" (OuterVolumeSpecName: "dshm") pod "6b543323-d070-43d7-aad8-41b6f1248a9c" (UID: "6b543323-d070-43d7-aad8-41b6f1248a9c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:50.958473 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.958382    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b543323-d070-43d7-aad8-41b6f1248a9c-kube-api-access-mcdmw" (OuterVolumeSpecName: "kube-api-access-mcdmw") pod "6b543323-d070-43d7-aad8-41b6f1248a9c" (UID: "6b543323-d070-43d7-aad8-41b6f1248a9c"). InnerVolumeSpecName "kube-api-access-mcdmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:51:50.958473 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:50.958389    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b543323-d070-43d7-aad8-41b6f1248a9c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b543323-d070-43d7-aad8-41b6f1248a9c" (UID: "6b543323-d070-43d7-aad8-41b6f1248a9c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:51:51.057062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.057040    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcdmw\" (UniqueName: \"kubernetes.io/projected/6b543323-d070-43d7-aad8-41b6f1248a9c-kube-api-access-mcdmw\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:51:51.057062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.057060    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b543323-d070-43d7-aad8-41b6f1248a9c-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:51:51.057192 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.057070    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:51:51.057192 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.057078    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:51:51.057192 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.057085    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b543323-d070-43d7-aad8-41b6f1248a9c-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:51:51.718434 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.718398    2562 generic.go:356] "Generic (PLEG): container finished" podID="6b543323-d070-43d7-aad8-41b6f1248a9c" containerID="30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9" exitCode=0
Sep 16 18:51:51.718900 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.718450    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"
Sep 16 18:51:51.718900 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.718478    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk" event={"ID":"6b543323-d070-43d7-aad8-41b6f1248a9c","Type":"ContainerDied","Data":"30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9"}
Sep 16 18:51:51.718900 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.718518    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk" event={"ID":"6b543323-d070-43d7-aad8-41b6f1248a9c","Type":"ContainerDied","Data":"64c7fed0ec5e5d4fd5d108c2e88ffb713710196b129b0c43b54cd30e6b8fe498"}
Sep 16 18:51:51.718900 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.718534    2562 scope.go:117] "RemoveContainer" containerID="30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9"
Sep 16 18:51:51.728832 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.728816    2562 scope.go:117] "RemoveContainer" containerID="30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9"
Sep 16 18:51:51.729097 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:51:51.729078    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9\": container with ID starting with 30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9 not found: ID does not exist" containerID="30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9"
Sep 16 18:51:51.729158 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.729107    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9"} err="failed to get container status \"30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9\": rpc error: code = NotFound desc = could not find container \"30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9\": container with ID starting with 30be527c5ff0c3263583f7418be2451e52ae69f26f637557044eae24b289c8d9 not found: ID does not exist"
Sep 16 18:51:51.740558 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.740534    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"]
Sep 16 18:51:51.744761 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:51.744734    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-6817d3/ipp-route-6817d3-kserve-578f7bd667-d67pk"]
Sep 16 18:51:52.285679 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:51:52.285646    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b543323-d070-43d7-aad8-41b6f1248a9c" path="/var/lib/kubelet/pods/6b543323-d070-43d7-aad8-41b6f1248a9c/volumes"
Sep 16 18:52:12.329370 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:12.329298    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:13.832911 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:13.832877    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:19.775395 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:19.775359    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:19.876311 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:19.876273    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:20.056408 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.056325    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:20.390433 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.390363    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"]
Sep 16 18:52:20.390736 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.390724    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b543323-d070-43d7-aad8-41b6f1248a9c" containerName="main"
Sep 16 18:52:20.390790 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.390737    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b543323-d070-43d7-aad8-41b6f1248a9c" containerName="main"
Sep 16 18:52:20.390790 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.390786    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13b95d3-ee2b-4e49-805c-01eb4a624109" containerName="revoke-keys"
Sep 16 18:52:20.390871 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.390792    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13b95d3-ee2b-4e49-805c-01eb4a624109" containerName="revoke-keys"
Sep 16 18:52:20.390871 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.390850    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b543323-d070-43d7-aad8-41b6f1248a9c" containerName="main"
Sep 16 18:52:20.390871 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.390858    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13b95d3-ee2b-4e49-805c-01eb4a624109" containerName="revoke-keys"
Sep 16 18:52:20.393950 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.393932    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"
Sep 16 18:52:20.396282 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.396261    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:52:20.396373 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.396349    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:52:20.402214 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.402173    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"]
Sep 16 18:52:20.582268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.582242    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb81c7c2-0798-46ad-980d-be39b81dc829-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz\" (UID: \"cb81c7c2-0798-46ad-980d-be39b81dc829\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"
Sep 16 18:52:20.683303 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.683278    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb81c7c2-0798-46ad-980d-be39b81dc829-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz\" (UID: \"cb81c7c2-0798-46ad-980d-be39b81dc829\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"
Sep 16 18:52:20.683833 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.683817    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb81c7c2-0798-46ad-980d-be39b81dc829-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz\" (UID: \"cb81c7c2-0798-46ad-980d-be39b81dc829\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"
Sep 16 18:52:20.704311 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.704295    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"
Sep 16 18:52:20.974602 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:20.974566    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"]
Sep 16 18:52:20.976764 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:52:20.976737    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb81c7c2_0798_46ad_980d_be39b81dc829.slice/crio-0c74babc4cc35d4cdd553e0cbdf6ed0ee5412d91e8b0e4b097baec5fb777a7cc WatchSource:0}: Error finding container 0c74babc4cc35d4cdd553e0cbdf6ed0ee5412d91e8b0e4b097baec5fb777a7cc: Status 404 returned error can't find the container with id 0c74babc4cc35d4cdd553e0cbdf6ed0ee5412d91e8b0e4b097baec5fb777a7cc
Sep 16 18:52:21.191281 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:21.191253    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:21.868841 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:21.868788    2562 generic.go:356] "Generic (PLEG): container finished" podID="cb81c7c2-0798-46ad-980d-be39b81dc829" containerID="5edc14cdef031bfd9d20bf5bf16225a50493cdb716bad9c114650be248ca1e75" exitCode=0
Sep 16 18:52:21.868939 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:21.868859    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz" event={"ID":"cb81c7c2-0798-46ad-980d-be39b81dc829","Type":"ContainerDied","Data":"5edc14cdef031bfd9d20bf5bf16225a50493cdb716bad9c114650be248ca1e75"}
Sep 16 18:52:21.868939 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:21.868913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz" event={"ID":"cb81c7c2-0798-46ad-980d-be39b81dc829","Type":"ContainerStarted","Data":"0c74babc4cc35d4cdd553e0cbdf6ed0ee5412d91e8b0e4b097baec5fb777a7cc"}
Sep 16 18:52:21.988079 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:21.988056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:22.673147 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:22.673120    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:23.005165 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:23.005139    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"
Sep 16 18:52:23.101763 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:23.101739    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb81c7c2-0798-46ad-980d-be39b81dc829-maas-api-service-ca\") pod \"cb81c7c2-0798-46ad-980d-be39b81dc829\" (UID: \"cb81c7c2-0798-46ad-980d-be39b81dc829\") "
Sep 16 18:52:23.102043 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:23.102022    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb81c7c2-0798-46ad-980d-be39b81dc829-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cb81c7c2-0798-46ad-980d-be39b81dc829" (UID: "cb81c7c2-0798-46ad-980d-be39b81dc829"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:52:23.202403 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:23.202378    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb81c7c2-0798-46ad-980d-be39b81dc829-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:52:23.461936 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:23.461912    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:23.879565 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:23.879500    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"
Sep 16 18:52:23.879565 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:23.879549    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz" event={"ID":"cb81c7c2-0798-46ad-980d-be39b81dc829","Type":"ContainerDied","Data":"0c74babc4cc35d4cdd553e0cbdf6ed0ee5412d91e8b0e4b097baec5fb777a7cc"}
Sep 16 18:52:23.879796 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:23.879601    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c74babc4cc35d4cdd553e0cbdf6ed0ee5412d91e8b0e4b097baec5fb777a7cc"
Sep 16 18:52:31.282323 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:31.282288    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:31.382998 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:31.382955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:31.487952 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:31.487923    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:31.570243 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:31.570164    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"]
Sep 16 18:52:31.572516 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:31.572495    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c90bd2-xbklz"]
Sep 16 18:52:31.575452 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:31.575429    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:31.637766 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:31.637739    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:31.999939 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:31.999908    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:32.101976 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:32.101938    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:32.285567 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:32.285509    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb81c7c2-0798-46ad-980d-be39b81dc829" path="/var/lib/kubelet/pods/cb81c7c2-0798-46ad-980d-be39b81dc829/volumes"
Sep 16 18:52:32.897139 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:32.897113    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:33.710026 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:33.709991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:38.116305 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.116270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:38.219757 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.219729    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:38.377286 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.377198    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:38.713926 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.713895    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"]
Sep 16 18:52:38.715010 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.714986    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb81c7c2-0798-46ad-980d-be39b81dc829" containerName="revoke-keys"
Sep 16 18:52:38.715149 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.715138    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb81c7c2-0798-46ad-980d-be39b81dc829" containerName="revoke-keys"
Sep 16 18:52:38.715430 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.715416    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb81c7c2-0798-46ad-980d-be39b81dc829" containerName="revoke-keys"
Sep 16 18:52:38.719731 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.719708    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"
Sep 16 18:52:38.722395 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.722184    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:52:38.722893 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.722871    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lfw6p\""
Sep 16 18:52:38.723498 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.723474    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"]
Sep 16 18:52:38.818746 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.818723    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ba9f255-38cd-4802-a64e-b346be23148d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8\" (UID: \"4ba9f255-38cd-4802-a64e-b346be23148d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"
Sep 16 18:52:38.920062 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.920038    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ba9f255-38cd-4802-a64e-b346be23148d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8\" (UID: \"4ba9f255-38cd-4802-a64e-b346be23148d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"
Sep 16 18:52:38.920630 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:38.920615    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ba9f255-38cd-4802-a64e-b346be23148d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8\" (UID: \"4ba9f255-38cd-4802-a64e-b346be23148d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"
Sep 16 18:52:39.030761 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:39.030709    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"
Sep 16 18:52:39.307266 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:39.307245    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"]
Sep 16 18:52:39.309206 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:52:39.309176    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ba9f255_38cd_4802_a64e_b346be23148d.slice/crio-c61acaf82f31ec890504310983ed7645a2ea8751e84c07de5e9f844b7ee7c840 WatchSource:0}: Error finding container c61acaf82f31ec890504310983ed7645a2ea8751e84c07de5e9f844b7ee7c840: Status 404 returned error can't find the container with id c61acaf82f31ec890504310983ed7645a2ea8751e84c07de5e9f844b7ee7c840
Sep 16 18:52:39.693356 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:39.693328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:39.957900 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:39.957879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8" event={"ID":"4ba9f255-38cd-4802-a64e-b346be23148d","Type":"ContainerStarted","Data":"0389289db0b5acaea614fac0d9724fa06b4f2f2d574f7885b6ed30910789a5ab"}
Sep 16 18:52:39.958008 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:39.957923    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8" event={"ID":"4ba9f255-38cd-4802-a64e-b346be23148d","Type":"ContainerStarted","Data":"c61acaf82f31ec890504310983ed7645a2ea8751e84c07de5e9f844b7ee7c840"}
Sep 16 18:52:39.974416 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:39.974381    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8" podStartSLOduration=1.974367601 podStartE2EDuration="1.974367601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:52:39.973298348 +0000 UTC m=+2180.213087100" watchObservedRunningTime="2026-09-16 18:52:39.974367601 +0000 UTC m=+2180.214156352"
Sep 16 18:52:40.501575 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:40.501549    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:40.964813 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:40.964785    2562 generic.go:356] "Generic (PLEG): container finished" podID="4ba9f255-38cd-4802-a64e-b346be23148d" containerID="0389289db0b5acaea614fac0d9724fa06b4f2f2d574f7885b6ed30910789a5ab" exitCode=0
Sep 16 18:52:40.964955 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:40.964823    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8" event={"ID":"4ba9f255-38cd-4802-a64e-b346be23148d","Type":"ContainerDied","Data":"0389289db0b5acaea614fac0d9724fa06b4f2f2d574f7885b6ed30910789a5ab"}
Sep 16 18:52:41.214693 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:41.214653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:42.099453 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:42.099432    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"
Sep 16 18:52:42.144949 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:42.144926    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ba9f255-38cd-4802-a64e-b346be23148d-maas-api-service-ca\") pod \"4ba9f255-38cd-4802-a64e-b346be23148d\" (UID: \"4ba9f255-38cd-4802-a64e-b346be23148d\") "
Sep 16 18:52:42.145231 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:42.145211    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ba9f255-38cd-4802-a64e-b346be23148d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ba9f255-38cd-4802-a64e-b346be23148d" (UID: "4ba9f255-38cd-4802-a64e-b346be23148d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:52:42.246122 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:42.246067    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ba9f255-38cd-4802-a64e-b346be23148d-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:52:42.976197 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:42.976166    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8" event={"ID":"4ba9f255-38cd-4802-a64e-b346be23148d","Type":"ContainerDied","Data":"c61acaf82f31ec890504310983ed7645a2ea8751e84c07de5e9f844b7ee7c840"}
Sep 16 18:52:42.976362 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:42.976239    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c61acaf82f31ec890504310983ed7645a2ea8751e84c07de5e9f844b7ee7c840"
Sep 16 18:52:42.976362 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:42.976178    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"
Sep 16 18:52:49.269866 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.269832    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:49.299804 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.299775    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"]
Sep 16 18:52:49.300034 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.300006    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6" podUID="34e3086f-ddc0-4550-94e4-da0e04a8c127" containerName="payload-processing" containerID="cri-o://9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2" gracePeriod=30
Sep 16 18:52:49.307397 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.307338    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"]
Sep 16 18:52:49.307569 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.307534    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt" podUID="c66061a2-cad0-4f3a-9c6a-83123e79badf" containerName="payload-pre-processing" containerID="cri-o://3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9" gracePeriod=30
Sep 16 18:52:49.347405 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.347373    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:49.411830 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.411798    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:49.481694 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.481653    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"]
Sep 16 18:52:49.487824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.487791    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-972df0-5nbx8"]
Sep 16 18:52:49.488026 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.487980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:49.552850 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.552813    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:49.579252 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.579228    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:52:49.581944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.581927    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:52:49.704492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.704467    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7nd4\" (UniqueName: \"kubernetes.io/projected/c66061a2-cad0-4f3a-9c6a-83123e79badf-kube-api-access-s7nd4\") pod \"c66061a2-cad0-4f3a-9c6a-83123e79badf\" (UID: \"c66061a2-cad0-4f3a-9c6a-83123e79badf\") "
Sep 16 18:52:49.704662 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.704495    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34e3086f-ddc0-4550-94e4-da0e04a8c127-plugins-config-volume\") pod \"34e3086f-ddc0-4550-94e4-da0e04a8c127\" (UID: \"34e3086f-ddc0-4550-94e4-da0e04a8c127\") "
Sep 16 18:52:49.704662 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.704562    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xp9wb\" (UniqueName: \"kubernetes.io/projected/34e3086f-ddc0-4550-94e4-da0e04a8c127-kube-api-access-xp9wb\") pod \"34e3086f-ddc0-4550-94e4-da0e04a8c127\" (UID: \"34e3086f-ddc0-4550-94e4-da0e04a8c127\") "
Sep 16 18:52:49.704662 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.704646    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c66061a2-cad0-4f3a-9c6a-83123e79badf-plugins-config-volume\") pod \"c66061a2-cad0-4f3a-9c6a-83123e79badf\" (UID: \"c66061a2-cad0-4f3a-9c6a-83123e79badf\") "
Sep 16 18:52:49.704898 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.704875    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34e3086f-ddc0-4550-94e4-da0e04a8c127-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34e3086f-ddc0-4550-94e4-da0e04a8c127" (UID: "34e3086f-ddc0-4550-94e4-da0e04a8c127"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:52:49.705057 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.705033    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c66061a2-cad0-4f3a-9c6a-83123e79badf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c66061a2-cad0-4f3a-9c6a-83123e79badf" (UID: "c66061a2-cad0-4f3a-9c6a-83123e79badf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:52:49.706728 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.706709    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c66061a2-cad0-4f3a-9c6a-83123e79badf-kube-api-access-s7nd4" (OuterVolumeSpecName: "kube-api-access-s7nd4") pod "c66061a2-cad0-4f3a-9c6a-83123e79badf" (UID: "c66061a2-cad0-4f3a-9c6a-83123e79badf"). InnerVolumeSpecName "kube-api-access-s7nd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:52:49.706816 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.706730    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e3086f-ddc0-4550-94e4-da0e04a8c127-kube-api-access-xp9wb" (OuterVolumeSpecName: "kube-api-access-xp9wb") pod "34e3086f-ddc0-4550-94e4-da0e04a8c127" (UID: "34e3086f-ddc0-4550-94e4-da0e04a8c127"). InnerVolumeSpecName "kube-api-access-xp9wb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:52:49.805260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.805210    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp9wb\" (UniqueName: \"kubernetes.io/projected/34e3086f-ddc0-4550-94e4-da0e04a8c127-kube-api-access-xp9wb\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:52:49.805260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.805231    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c66061a2-cad0-4f3a-9c6a-83123e79badf-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:52:49.805260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.805240    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7nd4\" (UniqueName: \"kubernetes.io/projected/c66061a2-cad0-4f3a-9c6a-83123e79badf-kube-api-access-s7nd4\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:52:49.805260 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.805248    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34e3086f-ddc0-4550-94e4-da0e04a8c127-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:52:49.878267 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.878236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:49.979139 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:49.979112    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:50.009344 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.009320    2562 generic.go:356] "Generic (PLEG): container finished" podID="c66061a2-cad0-4f3a-9c6a-83123e79badf" containerID="3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9" exitCode=0
Sep 16 18:52:50.009438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.009364    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"
Sep 16 18:52:50.009438 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.009411    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt" event={"ID":"c66061a2-cad0-4f3a-9c6a-83123e79badf","Type":"ContainerDied","Data":"3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9"}
Sep 16 18:52:50.009535 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.009486    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt" event={"ID":"c66061a2-cad0-4f3a-9c6a-83123e79badf","Type":"ContainerDied","Data":"1da1104f8cf0a601bf3220bb192549fe53acf73ef5fe5f6ec6f06f71570283f4"}
Sep 16 18:52:50.009535 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.009519    2562 scope.go:117] "RemoveContainer" containerID="3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9"
Sep 16 18:52:50.010738 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.010717    2562 generic.go:356] "Generic (PLEG): container finished" podID="34e3086f-ddc0-4550-94e4-da0e04a8c127" containerID="9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2" exitCode=0
Sep 16 18:52:50.010836 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.010764    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6" event={"ID":"34e3086f-ddc0-4550-94e4-da0e04a8c127","Type":"ContainerDied","Data":"9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2"}
Sep 16 18:52:50.010836 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.010798    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6" event={"ID":"34e3086f-ddc0-4550-94e4-da0e04a8c127","Type":"ContainerDied","Data":"a35fb3ff7f6024d8db965549622a011b20ed7eb36f8b4929897ef9428e9852cb"}
Sep 16 18:52:50.010836 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.010825    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"
Sep 16 18:52:50.019268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.019252    2562 scope.go:117] "RemoveContainer" containerID="3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9"
Sep 16 18:52:50.019557 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:52:50.019537    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9\": container with ID starting with 3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9 not found: ID does not exist" containerID="3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9"
Sep 16 18:52:50.019669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.019561    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9"} err="failed to get container status \"3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9\": rpc error: code = NotFound desc = could not find container \"3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9\": container with ID starting with 3fe35499d1af2fd3cc781e7c3519b48e2fbf7579600d5ddabc46090cf2f89bb9 not found: ID does not exist"
Sep 16 18:52:50.019669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.019576    2562 scope.go:117] "RemoveContainer" containerID="9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2"
Sep 16 18:52:50.027180 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.027165    2562 scope.go:117] "RemoveContainer" containerID="9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2"
Sep 16 18:52:50.027426 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:52:50.027411    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2\": container with ID starting with 9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2 not found: ID does not exist" containerID="9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2"
Sep 16 18:52:50.027492 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.027431    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2"} err="failed to get container status \"9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2\": rpc error: code = NotFound desc = could not find container \"9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2\": container with ID starting with 9948a9f8598ca18f4afd82ebdb7fe04ceee6687ee7b249e932c7fc441a7b9eb2 not found: ID does not exist"
Sep 16 18:52:50.034872 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.034853    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"]
Sep 16 18:52:50.038825 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.038806    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-972df0-76cd995c79-rkmd6"]
Sep 16 18:52:50.048685 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.048666    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"]
Sep 16 18:52:50.052037 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.052017    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-972df0-576dcd9cfd-b4zjt"]
Sep 16 18:52:50.285535 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.285515    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34e3086f-ddc0-4550-94e4-da0e04a8c127" path="/var/lib/kubelet/pods/34e3086f-ddc0-4550-94e4-da0e04a8c127/volumes"
Sep 16 18:52:50.285838 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.285831    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ba9f255-38cd-4802-a64e-b346be23148d" path="/var/lib/kubelet/pods/4ba9f255-38cd-4802-a64e-b346be23148d/volumes"
Sep 16 18:52:50.286126 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.286114    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c66061a2-cad0-4f3a-9c6a-83123e79badf" path="/var/lib/kubelet/pods/c66061a2-cad0-4f3a-9c6a-83123e79badf/volumes"
Sep 16 18:52:50.734432 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:50.734399    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:56.401638 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:56.401583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:56.500986 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:56.500943    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:56.744482 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:56.744456    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:57.319268 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:57.319225    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:58.315798 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:58.315759    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:58.987951 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:52:58.987923    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:07.533631 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.533555    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:07.562908 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.562878    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"]
Sep 16 18:53:07.563137 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.563105    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt" podUID="66ae44f1-788c-4729-a024-e1e0fcf35c05" containerName="payload-processing" containerID="cri-o://47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8" gracePeriod=30
Sep 16 18:53:07.570944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.570393    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"]
Sep 16 18:53:07.570944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.570614    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5" podUID="8ee3a65c-4999-4bfe-bcee-f81cfb502cdc" containerName="payload-pre-processing" containerID="cri-o://2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7" gracePeriod=30
Sep 16 18:53:07.650301 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.650270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:07.730817 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.730751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:07.797302 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.797229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:07.835510 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.835490    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:53:07.838567 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.838552    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:53:07.935771 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.935747    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-plugins-config-volume\") pod \"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc\" (UID: \"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc\") "
Sep 16 18:53:07.935918 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.935796    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbfln\" (UniqueName: \"kubernetes.io/projected/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-kube-api-access-mbfln\") pod \"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc\" (UID: \"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc\") "
Sep 16 18:53:07.935918 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.935818    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx7wk\" (UniqueName: \"kubernetes.io/projected/66ae44f1-788c-4729-a024-e1e0fcf35c05-kube-api-access-hx7wk\") pod \"66ae44f1-788c-4729-a024-e1e0fcf35c05\" (UID: \"66ae44f1-788c-4729-a024-e1e0fcf35c05\") "
Sep 16 18:53:07.935918 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.935895    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ae44f1-788c-4729-a024-e1e0fcf35c05-plugins-config-volume\") pod \"66ae44f1-788c-4729-a024-e1e0fcf35c05\" (UID: \"66ae44f1-788c-4729-a024-e1e0fcf35c05\") "
Sep 16 18:53:07.936112 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.936090    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ee3a65c-4999-4bfe-bcee-f81cfb502cdc" (UID: "8ee3a65c-4999-4bfe-bcee-f81cfb502cdc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:53:07.936342 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.936313    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66ae44f1-788c-4729-a024-e1e0fcf35c05-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66ae44f1-788c-4729-a024-e1e0fcf35c05" (UID: "66ae44f1-788c-4729-a024-e1e0fcf35c05"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:53:07.937991 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.937972    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66ae44f1-788c-4729-a024-e1e0fcf35c05-kube-api-access-hx7wk" (OuterVolumeSpecName: "kube-api-access-hx7wk") pod "66ae44f1-788c-4729-a024-e1e0fcf35c05" (UID: "66ae44f1-788c-4729-a024-e1e0fcf35c05"). InnerVolumeSpecName "kube-api-access-hx7wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:53:07.938065 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:07.938050    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-kube-api-access-mbfln" (OuterVolumeSpecName: "kube-api-access-mbfln") pod "8ee3a65c-4999-4bfe-bcee-f81cfb502cdc" (UID: "8ee3a65c-4999-4bfe-bcee-f81cfb502cdc"). InnerVolumeSpecName "kube-api-access-mbfln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:53:08.036529 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.036508    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ae44f1-788c-4729-a024-e1e0fcf35c05-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:53:08.036529 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.036528    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-plugins-config-volume\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:53:08.036669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.036538    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbfln\" (UniqueName: \"kubernetes.io/projected/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc-kube-api-access-mbfln\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:53:08.036669 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.036546    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx7wk\" (UniqueName: \"kubernetes.io/projected/66ae44f1-788c-4729-a024-e1e0fcf35c05-kube-api-access-hx7wk\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:53:08.100634 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.100555    2562 generic.go:356] "Generic (PLEG): container finished" podID="66ae44f1-788c-4729-a024-e1e0fcf35c05" containerID="47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8" exitCode=0
Sep 16 18:53:08.100634 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.100603    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"
Sep 16 18:53:08.100634 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.100632    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt" event={"ID":"66ae44f1-788c-4729-a024-e1e0fcf35c05","Type":"ContainerDied","Data":"47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8"}
Sep 16 18:53:08.100824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.100695    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt" event={"ID":"66ae44f1-788c-4729-a024-e1e0fcf35c05","Type":"ContainerDied","Data":"e074fcf7dc9b79d110a424c1c8b3bd9c366771f727f8d989a0a795a37388492d"}
Sep 16 18:53:08.100824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.100715    2562 scope.go:117] "RemoveContainer" containerID="47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8"
Sep 16 18:53:08.102150 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.102123    2562 generic.go:356] "Generic (PLEG): container finished" podID="8ee3a65c-4999-4bfe-bcee-f81cfb502cdc" containerID="2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7" exitCode=0
Sep 16 18:53:08.102252 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.102155    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"
Sep 16 18:53:08.102252 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.102177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5" event={"ID":"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc","Type":"ContainerDied","Data":"2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7"}
Sep 16 18:53:08.102252 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.102216    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5" event={"ID":"8ee3a65c-4999-4bfe-bcee-f81cfb502cdc","Type":"ContainerDied","Data":"a95a28cadf17bb0b97a4b7a4b65633249361b417d49476c560e68be91077f0a5"}
Sep 16 18:53:08.109751 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.109734    2562 scope.go:117] "RemoveContainer" containerID="47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8"
Sep 16 18:53:08.110026 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:53:08.109996    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8\": container with ID starting with 47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8 not found: ID does not exist" containerID="47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8"
Sep 16 18:53:08.110094 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.110032    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8"} err="failed to get container status \"47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8\": rpc error: code = NotFound desc = could not find container \"47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8\": container with ID starting with 47db65cef18d48782121a48b2c50bff90462160a9eeeed2b9796b124db6db9e8 not found: ID does not exist"
Sep 16 18:53:08.110094 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.110048    2562 scope.go:117] "RemoveContainer" containerID="2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7"
Sep 16 18:53:08.114537 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.114500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:08.117975 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.117959    2562 scope.go:117] "RemoveContainer" containerID="2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7"
Sep 16 18:53:08.118203 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:53:08.118188    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7\": container with ID starting with 2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7 not found: ID does not exist" containerID="2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7"
Sep 16 18:53:08.118245 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.118210    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7"} err="failed to get container status \"2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7\": rpc error: code = NotFound desc = could not find container \"2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7\": container with ID starting with 2e2166a253f06abc91c70dd03964020fea72c8eb7628e5d5911f8d6f33b527f7 not found: ID does not exist"
Sep 16 18:53:08.121753 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.121734    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"]
Sep 16 18:53:08.125799 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.125777    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-d3e8be-55f479f8c9-nnczt"]
Sep 16 18:53:08.135294 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.135272    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"]
Sep 16 18:53:08.139840 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.139821    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d3e8be-78dc748746-f4zg5"]
Sep 16 18:53:08.214992 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.214961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:08.284824 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.284803    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66ae44f1-788c-4729-a024-e1e0fcf35c05" path="/var/lib/kubelet/pods/66ae44f1-788c-4729-a024-e1e0fcf35c05/volumes"
Sep 16 18:53:08.285123 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:08.285111    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ee3a65c-4999-4bfe-bcee-f81cfb502cdc" path="/var/lib/kubelet/pods/8ee3a65c-4999-4bfe-bcee-f81cfb502cdc/volumes"
Sep 16 18:53:14.694274 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:14.694231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:14.761027 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:14.760984    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:14.992356 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:14.992271    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:18.691627 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:18.691584    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:57.081480 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:57.081449    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:57.161381 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:53:57.161347    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:19.077665 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:54:19.077613    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:40.412907 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:54:40.412874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:42.291350 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:54:42.291316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:33.826699 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:56:33.826657    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:53.351325 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:56:53.351250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:04.424575 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:04.424543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:20.845960 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:20.845926    2562 scope.go:117] "RemoveContainer" containerID="4fc2c23e53ad45a0a74c5f7bb3ebd9f5e6ba0b25b0140444c25ae34c37f18947"
Sep 16 18:57:20.854398 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:20.854361    2562 scope.go:117] "RemoveContainer" containerID="c9d7142fc27eaf11d277eff25ff352c77b0a5ff3fbade998c47dc223e7bc700f"
Sep 16 18:57:37.155240 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:37.155203    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:46.788537 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:46.788508    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:48.233535 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:48.233502    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:51.116525 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.116491    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"]
Sep 16 18:57:51.116895 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.116874    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c66061a2-cad0-4f3a-9c6a-83123e79badf" containerName="payload-pre-processing"
Sep 16 18:57:51.116895 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.116886    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c66061a2-cad0-4f3a-9c6a-83123e79badf" containerName="payload-pre-processing"
Sep 16 18:57:51.116971 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.116897    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ba9f255-38cd-4802-a64e-b346be23148d" containerName="revoke-keys"
Sep 16 18:57:51.116971 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.116902    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ba9f255-38cd-4802-a64e-b346be23148d" containerName="revoke-keys"
Sep 16 18:57:51.116971 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.116910    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ee3a65c-4999-4bfe-bcee-f81cfb502cdc" containerName="payload-pre-processing"
Sep 16 18:57:51.116971 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.116915    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ee3a65c-4999-4bfe-bcee-f81cfb502cdc" containerName="payload-pre-processing"
Sep 16 18:57:51.116971 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.116922    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34e3086f-ddc0-4550-94e4-da0e04a8c127" containerName="payload-processing"
Sep 16 18:57:51.116971 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.116927    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="34e3086f-ddc0-4550-94e4-da0e04a8c127" containerName="payload-processing"
Sep 16 18:57:51.116971 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.116937    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66ae44f1-788c-4729-a024-e1e0fcf35c05" containerName="payload-processing"
Sep 16 18:57:51.116971 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.116942    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="66ae44f1-788c-4729-a024-e1e0fcf35c05" containerName="payload-processing"
Sep 16 18:57:51.117205 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.117003    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c66061a2-cad0-4f3a-9c6a-83123e79badf" containerName="payload-pre-processing"
Sep 16 18:57:51.117205 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.117013    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="34e3086f-ddc0-4550-94e4-da0e04a8c127" containerName="payload-processing"
Sep 16 18:57:51.117205 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.117017    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ee3a65c-4999-4bfe-bcee-f81cfb502cdc" containerName="payload-pre-processing"
Sep 16 18:57:51.117205 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.117023    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ba9f255-38cd-4802-a64e-b346be23148d" containerName="revoke-keys"
Sep 16 18:57:51.117205 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.117029    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="66ae44f1-788c-4729-a024-e1e0fcf35c05" containerName="payload-processing"
Sep 16 18:57:51.119916 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.119895    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.130387 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.130366    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-624f44dc-kserve-self-signed-certs\""
Sep 16 18:57:51.133016 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.132996    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"]
Sep 16 18:57:51.157338 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.157313    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:51.243326 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.243305    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjmq9\" (UniqueName: \"kubernetes.io/projected/bf63750b-8e75-4562-8944-380215127910-kube-api-access-gjmq9\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.243447 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.243353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-home\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.243447 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.243378    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-model-cache\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.243447 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.243409    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf63750b-8e75-4562-8944-380215127910-tls-certs\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.243631 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.243465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-dshm\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.243631 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.243518    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-tmp-dir\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.344369 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.344345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-tmp-dir\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.344466 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.344387    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjmq9\" (UniqueName: \"kubernetes.io/projected/bf63750b-8e75-4562-8944-380215127910-kube-api-access-gjmq9\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.344466 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.344411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-home\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.344466 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.344424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-model-cache\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.344637 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.344543    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf63750b-8e75-4562-8944-380215127910-tls-certs\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.344698 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.344644    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-dshm\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.344783 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.344765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-model-cache\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.344836 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.344821    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-home\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.344892 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.344874    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-tmp-dir\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.346902 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.346879    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-dshm\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.347009 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.346984    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf63750b-8e75-4562-8944-380215127910-tls-certs\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.352812 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.352795    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjmq9\" (UniqueName: \"kubernetes.io/projected/bf63750b-8e75-4562-8944-380215127910-kube-api-access-gjmq9\") pod \"e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") " pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.428070 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.428051    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:57:51.706070 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:51.705988    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"]
Sep 16 18:57:51.709461 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:57:51.709434    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf63750b_8e75_4562_8944_380215127910.slice/crio-217a220e532ae335d68bd639573865a1c697bc367255ccd628af419edcc69c60 WatchSource:0}: Error finding container 217a220e532ae335d68bd639573865a1c697bc367255ccd628af419edcc69c60: Status 404 returned error can't find the container with id 217a220e532ae335d68bd639573865a1c697bc367255ccd628af419edcc69c60
Sep 16 18:57:52.412944 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:52.412911    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq" event={"ID":"bf63750b-8e75-4562-8944-380215127910","Type":"ContainerStarted","Data":"6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b"}
Sep 16 18:57:52.413303 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:52.412951    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq" event={"ID":"bf63750b-8e75-4562-8944-380215127910","Type":"ContainerStarted","Data":"217a220e532ae335d68bd639573865a1c697bc367255ccd628af419edcc69c60"}
Sep 16 18:57:52.480755 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:57:52.480706    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq" podStartSLOduration=1.4806915219999999 podStartE2EDuration="1.480691522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:57:52.479088783 +0000 UTC m=+2492.718877535" watchObservedRunningTime="2026-09-16 18:57:52.480691522 +0000 UTC m=+2492.720480273"
Sep 16 18:58:01.429285 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:01.429253    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:58:01.429771 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:01.429294    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:58:01.441439 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:01.441417    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:58:01.464007 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:01.463988    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:58:03.952362 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:03.952319    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:05.197162 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:05.197132    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"]
Sep 16 18:58:05.219811 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:05.219786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:20.216069 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.216017    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq" podUID="bf63750b-8e75-4562-8944-380215127910" containerName="main" containerID="cri-o://6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b" gracePeriod=45
Sep 16 18:58:20.459145 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.459124    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:58:20.550957 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.550886    2562 generic.go:356] "Generic (PLEG): container finished" podID="bf63750b-8e75-4562-8944-380215127910" containerID="6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b" exitCode=0
Sep 16 18:58:20.550957 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.550932    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"
Sep 16 18:58:20.551165 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.550972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq" event={"ID":"bf63750b-8e75-4562-8944-380215127910","Type":"ContainerDied","Data":"6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b"}
Sep 16 18:58:20.551165 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.551013    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq" event={"ID":"bf63750b-8e75-4562-8944-380215127910","Type":"ContainerDied","Data":"217a220e532ae335d68bd639573865a1c697bc367255ccd628af419edcc69c60"}
Sep 16 18:58:20.551165 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.551026    2562 scope.go:117] "RemoveContainer" containerID="6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b"
Sep 16 18:58:20.556317 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556295    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-tmp-dir\") pod \"bf63750b-8e75-4562-8944-380215127910\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") "
Sep 16 18:58:20.556441 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556329    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-dshm\") pod \"bf63750b-8e75-4562-8944-380215127910\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") "
Sep 16 18:58:20.556441 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556394    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjmq9\" (UniqueName: \"kubernetes.io/projected/bf63750b-8e75-4562-8944-380215127910-kube-api-access-gjmq9\") pod \"bf63750b-8e75-4562-8944-380215127910\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") "
Sep 16 18:58:20.556441 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556416    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf63750b-8e75-4562-8944-380215127910-tls-certs\") pod \"bf63750b-8e75-4562-8944-380215127910\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") "
Sep 16 18:58:20.556619 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556530    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf63750b-8e75-4562-8944-380215127910" (UID: "bf63750b-8e75-4562-8944-380215127910"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:20.556619 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556542    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-model-cache\") pod \"bf63750b-8e75-4562-8944-380215127910\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") "
Sep 16 18:58:20.556724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556623    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-home\") pod \"bf63750b-8e75-4562-8944-380215127910\" (UID: \"bf63750b-8e75-4562-8944-380215127910\") "
Sep 16 18:58:20.556776 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556737    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf63750b-8e75-4562-8944-380215127910" (UID: "bf63750b-8e75-4562-8944-380215127910"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:20.556848 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556830    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-home" (OuterVolumeSpecName: "home") pod "bf63750b-8e75-4562-8944-380215127910" (UID: "bf63750b-8e75-4562-8944-380215127910"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:20.556985 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556970    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:58:20.557068 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556986    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:58:20.557068 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.556999    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:58:20.558631 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.558608    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-dshm" (OuterVolumeSpecName: "dshm") pod "bf63750b-8e75-4562-8944-380215127910" (UID: "bf63750b-8e75-4562-8944-380215127910"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:20.559071 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.559052    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf63750b-8e75-4562-8944-380215127910-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf63750b-8e75-4562-8944-380215127910" (UID: "bf63750b-8e75-4562-8944-380215127910"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:58:20.559137 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.559066    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf63750b-8e75-4562-8944-380215127910-kube-api-access-gjmq9" (OuterVolumeSpecName: "kube-api-access-gjmq9") pod "bf63750b-8e75-4562-8944-380215127910" (UID: "bf63750b-8e75-4562-8944-380215127910"). InnerVolumeSpecName "kube-api-access-gjmq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:58:20.561405 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.561332    2562 scope.go:117] "RemoveContainer" containerID="6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b"
Sep 16 18:58:20.561815 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:58:20.561754    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b\": container with ID starting with 6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b not found: ID does not exist" containerID="6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b"
Sep 16 18:58:20.561815 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.561786    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b"} err="failed to get container status \"6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b\": rpc error: code = NotFound desc = could not find container \"6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b\": container with ID starting with 6ba86a61168a49de07769b4fe014ef7baf19a740db0805d4b4de9e137eaf1d1b not found: ID does not exist"
Sep 16 18:58:20.657665 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.657645    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf63750b-8e75-4562-8944-380215127910-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:58:20.657665 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.657664    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjmq9\" (UniqueName: \"kubernetes.io/projected/bf63750b-8e75-4562-8944-380215127910-kube-api-access-gjmq9\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:58:20.657784 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.657673    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf63750b-8e75-4562-8944-380215127910-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\""
Sep 16 18:58:20.877161 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.877136    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"]
Sep 16 18:58:20.885343 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:20.885326    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-624f44dc-kserve-cc75bbb75-6m9rq"]
Sep 16 18:58:22.284939 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:22.284908    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf63750b-8e75-4562-8944-380215127910" path="/var/lib/kubelet/pods/bf63750b-8e75-4562-8944-380215127910/volumes"
Sep 16 18:58:54.526174 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:58:54.526143    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f57666fc5-4mmlh_33c1c5fa-c340-4dc8-9478-0108e4eef38c/authorino/0.log"
Sep 16 18:59:01.072142 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:01.072073    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bdbbf975-2cjqf_4187fb1c-8176-4c0f-82bb-ed212c9c1d41/copy-manifests/0.log"
Sep 16 18:59:01.089396 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:01.089377    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bdbbf975-2cjqf_4187fb1c-8176-4c0f-82bb-ed212c9c1d41/manager/0.log"
Sep 16 18:59:02.059471 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:02.059447    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29826405-k8hzw_a094a321-6f0f-4719-af2c-d9dc6ca8bc24/cleanup/0.log"
Sep 16 18:59:02.998026 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:02.998002    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f57666fc5-4mmlh_33c1c5fa-c340-4dc8-9478-0108e4eef38c/authorino/0.log"
Sep 16 18:59:03.399996 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:03.399935    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-msrll_84176ad5-47ab-45c8-af4b-e001945f01a7/kuadrant-console-plugin/0.log"
Sep 16 18:59:04.700627 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:04.700603    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d9c67857d-p97jv_a4436fe7-191c-4c7d-b381-e7525adb6680/kube-auth-proxy/0.log"
Sep 16 18:59:04.949873 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:04.949846    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ss6mc_dd0d133b-b605-4231-9399-72b5db35bb51/payload-pre-processing/0.log"
Sep 16 18:59:05.080920 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:05.080859    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pjwhq_1963517c-bd16-44cd-9994-8a432db84f92/payload-processing/0.log"
Sep 16 18:59:05.207175 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:05.207153    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fcd5855b4-c2lsw_64086446-7f12-484e-b1e0-c0f57190258e/router/0.log"
Sep 16 18:59:05.566319 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:05.566296    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-5fszv_3d5b4998-680b-45c0-a4ad-470066799f65/main/0.log"
Sep 16 18:59:05.923439 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:05.923420    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mwztf_f70cee47-467a-43ba-8a4a-ac183fa2d74e/main/0.log"
Sep 16 18:59:06.299410 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:06.299361    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-qcl6m_80594d49-9246-423f-8411-9a52f079d218/main/0.log"
Sep 16 18:59:11.549501 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:11.549470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:11.709027 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:11.709002    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:15.112272 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:15.112245    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2x4vg_8b056af3-46bd-4340-8336-66d3e52a40f7/global-pull-secret-syncer/0.log"
Sep 16 18:59:15.508495 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:15.508471    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4wchr_a820827f-2aeb-4415-b031-2278cde1f521/konnectivity-agent/0.log"
Sep 16 18:59:15.825778 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:15.825717    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-100.ec2.internal_35aa6f7c2ee614a3c7e88ad95244c3eb/haproxy/0.log"
Sep 16 18:59:19.783701 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:19.783672    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f57666fc5-4mmlh_33c1c5fa-c340-4dc8-9478-0108e4eef38c/authorino/0.log"
Sep 16 18:59:20.076282 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:20.076214    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-msrll_84176ad5-47ab-45c8-af4b-e001945f01a7/kuadrant-console-plugin/0.log"
Sep 16 18:59:20.887633 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:20.887581    2562 scope.go:117] "RemoveContainer" containerID="5edc14cdef031bfd9d20bf5bf16225a50493cdb716bad9c114650be248ca1e75"
Sep 16 18:59:20.895427 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:20.895408    2562 scope.go:117] "RemoveContainer" containerID="0389289db0b5acaea614fac0d9724fa06b4f2f2d574f7885b6ed30910789a5ab"
Sep 16 18:59:22.234496 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:22.234472    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-9lhhj_23bc5849-687e-4a9d-9eab-10899639bb62/kube-state-metrics/0.log"
Sep 16 18:59:22.276399 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:22.276379    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-9lhhj_23bc5849-687e-4a9d-9eab-10899639bb62/kube-rbac-proxy-main/0.log"
Sep 16 18:59:22.335289 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:22.335273    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-9lhhj_23bc5849-687e-4a9d-9eab-10899639bb62/kube-rbac-proxy-self/0.log"
Sep 16 18:59:22.575047 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:22.574992    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4p87f_82ea4326-263a-4fd2-bd4b-3c76f0b874fb/node-exporter/0.log"
Sep 16 18:59:22.602958 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:22.602937    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4p87f_82ea4326-263a-4fd2-bd4b-3c76f0b874fb/kube-rbac-proxy/0.log"
Sep 16 18:59:22.639193 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:22.639178    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4p87f_82ea4326-263a-4fd2-bd4b-3c76f0b874fb/init-textfile/0.log"
Sep 16 18:59:22.769384 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:22.769363    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7lf6t_f26f53fe-ffed-4063-bac9-b569e5fb12dc/kube-rbac-proxy-main/0.log"
Sep 16 18:59:22.803063 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:22.803041    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7lf6t_f26f53fe-ffed-4063-bac9-b569e5fb12dc/kube-rbac-proxy-self/0.log"
Sep 16 18:59:22.852255 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:22.852209    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7lf6t_f26f53fe-ffed-4063-bac9-b569e5fb12dc/openshift-state-metrics/0.log"
Sep 16 18:59:23.401567 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:23.401546    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b8cfbb97b-twrv5_7f591335-8f7c-41d5-b9a1-4ab18335b7b9/telemeter-client/0.log"
Sep 16 18:59:23.444020 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:23.443998    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b8cfbb97b-twrv5_7f591335-8f7c-41d5-b9a1-4ab18335b7b9/reload/0.log"
Sep 16 18:59:23.489463 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:23.489444    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b8cfbb97b-twrv5_7f591335-8f7c-41d5-b9a1-4ab18335b7b9/kube-rbac-proxy/0.log"
Sep 16 18:59:23.544435 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:23.544412    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668478c48-cz8dl_9f867486-44dc-4c83-aca6-1ce2e12091b8/thanos-query/0.log"
Sep 16 18:59:23.581734 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:23.581716    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668478c48-cz8dl_9f867486-44dc-4c83-aca6-1ce2e12091b8/kube-rbac-proxy-web/0.log"
Sep 16 18:59:23.618956 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:23.618941    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668478c48-cz8dl_9f867486-44dc-4c83-aca6-1ce2e12091b8/kube-rbac-proxy/0.log"
Sep 16 18:59:23.659120 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:23.659077    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668478c48-cz8dl_9f867486-44dc-4c83-aca6-1ce2e12091b8/prom-label-proxy/0.log"
Sep 16 18:59:23.710226 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:23.710207    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668478c48-cz8dl_9f867486-44dc-4c83-aca6-1ce2e12091b8/kube-rbac-proxy-rules/0.log"
Sep 16 18:59:23.752080 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:23.752064    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668478c48-cz8dl_9f867486-44dc-4c83-aca6-1ce2e12091b8/kube-rbac-proxy-metrics/0.log"
Sep 16 18:59:24.230681 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.230655    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"]
Sep 16 18:59:24.231030 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.231017    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf63750b-8e75-4562-8944-380215127910" containerName="main"
Sep 16 18:59:24.231086 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.231031    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf63750b-8e75-4562-8944-380215127910" containerName="main"
Sep 16 18:59:24.231121 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.231096    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf63750b-8e75-4562-8944-380215127910" containerName="main"
Sep 16 18:59:24.235651 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.235637    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.240618 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:59:24.240536    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-must-gather-slrm4\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-must-gather-slrm4\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 16 18:59:24.240618 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:59:24.240572    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-bzh68\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-must-gather-slrm4\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-must-gather-slrm4\"/\"default-dockercfg-bzh68\"" type="*v1.Secret"
Sep 16 18:59:24.240769 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:59:24.240628    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"perf-node-gather-daemonset-s9wpz\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-slrm4\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="f69c0b1c-b95a-4124-9722-67d01c5c3eb1" pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.240769 ip-10-0-133-100 kubenswrapper[2562]: E0916 18:59:24.240671    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-must-gather-slrm4\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-must-gather-slrm4\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 16 18:59:24.258834 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.258811    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"]
Sep 16 18:59:24.429538 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.429505    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-podres\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.429874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.429547    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-proc\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.429874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.429625    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-lib-modules\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.429874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.429654    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-sys\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.429874 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.429705    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtk9g\" (UniqueName: \"kubernetes.io/projected/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-kube-api-access-jtk9g\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.530778 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.530719    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-lib-modules\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.530778 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.530746    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-sys\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.530778 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.530776    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtk9g\" (UniqueName: \"kubernetes.io/projected/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-kube-api-access-jtk9g\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.531029 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.530802    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-podres\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.531029 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.530830    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-proc\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.531029 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.530854    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-sys\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.531029 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.530899    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-lib-modules\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.531029 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.530916    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-podres\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:24.531029 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:24.530925    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-proc\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:25.088921 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:25.088900    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-slrm4\"/\"kube-root-ca.crt\""
Sep 16 18:59:25.678911 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:25.678889    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-slrm4\"/\"openshift-service-ca.crt\""
Sep 16 18:59:25.683227 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:25.683206    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtk9g\" (UniqueName: \"kubernetes.io/projected/f69c0b1c-b95a-4124-9722-67d01c5c3eb1-kube-api-access-jtk9g\") pod \"perf-node-gather-daemonset-s9wpz\" (UID: \"f69c0b1c-b95a-4124-9722-67d01c5c3eb1\") " pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:25.724742 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:25.724715    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-slrm4\"/\"default-dockercfg-bzh68\""
Sep 16 18:59:25.747869 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:25.747834    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:26.023679 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:26.023653    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"]
Sep 16 18:59:26.025514 ip-10-0-133-100 kubenswrapper[2562]: W0916 18:59:26.025489    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf69c0b1c_b95a_4124_9722_67d01c5c3eb1.slice/crio-1682e3613b544921b578392ac1d7ffc8e0ad1787bcb8fb5a2f01bbbca89ca84f WatchSource:0}: Error finding container 1682e3613b544921b578392ac1d7ffc8e0ad1787bcb8fb5a2f01bbbca89ca84f: Status 404 returned error can't find the container with id 1682e3613b544921b578392ac1d7ffc8e0ad1787bcb8fb5a2f01bbbca89ca84f
Sep 16 18:59:26.468692 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:26.468668    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-66fb54f4b8-xl7gs_4d7c3815-73b2-408b-9525-560443edd089/console/0.log"
Sep 16 18:59:26.509120 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:26.509100    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-zvj95_24c6189d-c787-49c4-9a04-1617da5ad6de/download-server/0.log"
Sep 16 18:59:26.861724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:26.861661    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz" event={"ID":"f69c0b1c-b95a-4124-9722-67d01c5c3eb1","Type":"ContainerStarted","Data":"1a916434c8d7cde61d7b29409e354b58397db058eb1dc8d2f2c90591472822f0"}
Sep 16 18:59:26.861724 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:26.861718    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz" event={"ID":"f69c0b1c-b95a-4124-9722-67d01c5c3eb1","Type":"ContainerStarted","Data":"1682e3613b544921b578392ac1d7ffc8e0ad1787bcb8fb5a2f01bbbca89ca84f"}
Sep 16 18:59:26.862052 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:26.861748    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:26.888744 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:26.888705    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz" podStartSLOduration=2.888690758 podStartE2EDuration="2.888690758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:59:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:59:26.888061964 +0000 UTC m=+2587.127850716" watchObservedRunningTime="2026-09-16 18:59:26.888690758 +0000 UTC m=+2587.128479509"
Sep 16 18:59:27.285606 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:27.285573    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-qnrm5_6f83ab39-0687-4870-a4d6-2cb1aaf4a0e1/volume-data-source-validator/0.log"
Sep 16 18:59:28.542601 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:28.542573    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4dzgz_bc29fcde-2de7-4474-beff-440b1e1724cf/dns/0.log"
Sep 16 18:59:28.583297 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:28.583276    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4dzgz_bc29fcde-2de7-4474-beff-440b1e1724cf/kube-rbac-proxy/0.log"
Sep 16 18:59:28.883915 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:28.883866    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fxbpm_634753ab-bf55-4898-a91d-7b08037e8ac3/dns-node-resolver/0.log"
Sep 16 18:59:29.669746 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:29.669723    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7d65f57994-7qdks_cebccbde-4c2f-4e98-ac48-15aad4cc6317/registry/0.log"
Sep 16 18:59:29.792527 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:29.792501    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gzdnv_a094404a-9871-499f-bbf5-0223671705d4/node-ca/0.log"
Sep 16 18:59:31.439626 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:31.439580    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d9c67857d-p97jv_a4436fe7-191c-4c7d-b381-e7525adb6680/kube-auth-proxy/0.log"
Sep 16 18:59:31.531338 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:31.531317    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ss6mc_dd0d133b-b605-4231-9399-72b5db35bb51/payload-pre-processing/0.log"
Sep 16 18:59:31.580523 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:31.580499    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pjwhq_1963517c-bd16-44cd-9994-8a432db84f92/payload-processing/0.log"
Sep 16 18:59:31.622429 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:31.622408    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fcd5855b4-c2lsw_64086446-7f12-484e-b1e0-c0f57190258e/router/0.log"
Sep 16 18:59:32.776139 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:32.776115    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q4tb8_1ea3df26-130f-4aeb-a1e9-527f755670d5/serve-healthcheck-canary/0.log"
Sep 16 18:59:32.876026 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:32.876007    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-slrm4/perf-node-gather-daemonset-s9wpz"
Sep 16 18:59:33.630934 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:33.630912    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-42dj7_56254d14-1b92-4b27-aa73-678c951f78ef/insights-operator/0.log"
Sep 16 18:59:33.631165 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:33.631142    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-42dj7_56254d14-1b92-4b27-aa73-678c951f78ef/insights-operator/1.log"
Sep 16 18:59:34.155677 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:34.155654    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s8brz_572ee141-d531-4716-b163-0f3c38d55648/kube-rbac-proxy/0.log"
Sep 16 18:59:34.231078 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:34.231053    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s8brz_572ee141-d531-4716-b163-0f3c38d55648/exporter/0.log"
Sep 16 18:59:34.282194 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:34.282170    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s8brz_572ee141-d531-4716-b163-0f3c38d55648/extractor/0.log"
Sep 16 18:59:35.306535 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:35.306516    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d9c67857d-p97jv_a4436fe7-191c-4c7d-b381-e7525adb6680/kube-auth-proxy/0.log"
Sep 16 18:59:35.387386 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:35.387365    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ss6mc_dd0d133b-b605-4231-9399-72b5db35bb51/payload-pre-processing/0.log"
Sep 16 18:59:35.422981 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:35.422950    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pjwhq_1963517c-bd16-44cd-9994-8a432db84f92/payload-processing/0.log"
Sep 16 18:59:35.467132 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:35.467111    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fcd5855b4-c2lsw_64086446-7f12-484e-b1e0-c0f57190258e/router/0.log"
Sep 16 18:59:37.787634 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:37.787608    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d9c67857d-p97jv_a4436fe7-191c-4c7d-b381-e7525adb6680/kube-auth-proxy/0.log"
Sep 16 18:59:37.864004 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:37.863945    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ss6mc_dd0d133b-b605-4231-9399-72b5db35bb51/payload-pre-processing/0.log"
Sep 16 18:59:37.915705 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:37.915683    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-pjwhq_1963517c-bd16-44cd-9994-8a432db84f92/payload-processing/0.log"
Sep 16 18:59:37.940577 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:37.940556    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fcd5855b4-c2lsw_64086446-7f12-484e-b1e0-c0f57190258e/router/0.log"
Sep 16 18:59:38.091239 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:38.091216    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bdbbf975-2cjqf_4187fb1c-8176-4c0f-82bb-ed212c9c1d41/manager/0.log"
Sep 16 18:59:38.139485 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:38.139433    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86bdbbf975-2cjqf_4187fb1c-8176-4c0f-82bb-ed212c9c1d41/copy-manifests/0.log"
Sep 16 18:59:39.812736 ip-10-0-133-100 kubenswrapper[2562]: I0916 18:59:39.812704    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6fb494f6fb-khrnj_15b621ee-8544-44b2-b121-c326440878d4/manager/0.log"