Jul 29 17:50:24.909072 ip-10-0-135-253 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 17:50:24.909082 ip-10-0-135-253 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 17:50:24.909089 ip-10-0-135-253 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 17:50:24.909294 ip-10-0-135-253 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 17:50:34.923064 ip-10-0-135-253 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 17:50:34.923081 ip-10-0-135-253 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e75c3059b8e04d3f93c782ea1f14d6ff --
Jul 29 17:52:10.364653 ip-10-0-135-253 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 17:52:10.799454 ip-10-0-135-253 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:52:10.799454 ip-10-0-135-253 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 17:52:10.799454 ip-10-0-135-253 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:52:10.799454 ip-10-0-135-253 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 17:52:10.799454 ip-10-0-135-253 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:52:10.802577 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.802489    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 17:52:10.807468 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807451    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:10.807468 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807468    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807472    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807475    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807479    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807482    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807485    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807488    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807491    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807494    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807497    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807499    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807502    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807504    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807507    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807509    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807512    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807515    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807517    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807520    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:10.807532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807526    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807529    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807532    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807534    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807537    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807540    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807543    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807546    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807548    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807551    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807556    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807561    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807564    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807567    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807570    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807573    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807576    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807578    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807581    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:10.807985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807584    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807586    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807588    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807591    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807594    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807596    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807599    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807602    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807604    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807607    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807610    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807612    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807614    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807617    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807620    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807624    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807627    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807629    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807632    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807635    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:10.808470 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807637    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807640    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807642    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807645    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807648    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807650    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807653    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807656    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807660    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807664    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807667    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807670    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807672    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807675    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807678    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807680    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807683    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807686    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807688    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:10.808985 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807691    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807693    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807696    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807699    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807701    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807704    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807706    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807709    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807713    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807716    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807719    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807723    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807725    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807728    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807730    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807733    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807735    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807738    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807740    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807742    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:10.809462 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.807745    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:10.811298 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811277    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:10.811298 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811294    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:10.811298 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811298    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:10.811298 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811301    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:10.811298 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811305    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811308    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811311    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811314    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811317    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811320    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811343    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811348    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811352    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811355    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811357    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811371    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811374    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811377    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811380    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811383    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811387    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811390    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811393    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:10.811515 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811396    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811399    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811401    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811404    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811408    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811412    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811415    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811418    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811423    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811425    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811428    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811431    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811433    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811437    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811440    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811443    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811446    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811449    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811451    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811454    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:10.811992 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811456    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811459    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811462    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811465    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811467    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811472    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811476    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811479    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811482    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811485    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811489    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811492    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811494    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811497    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811500    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811502    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811505    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811507    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811510    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:10.812492 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811512    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811515    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811517    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811520    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811523    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811525    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811528    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811531    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811535    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811537    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811540    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811543    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811545    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811548    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811550    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811553    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811555    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811558    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811560    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811563    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:10.812967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811565    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811569    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811571    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811574    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811577    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811579    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811581    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811584    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811587    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811589    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811591    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811594    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811597    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811599    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811602    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811604    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811607    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.811996    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812000    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812003    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:10.813481 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812007    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812009    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812012    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812015    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812018    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812020    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812023    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812026    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812029    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812031    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812034    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812037    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812041    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812045    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812048    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812051    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812054    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812056    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812059    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:10.813973 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812061    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812065    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812067    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812071    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812074    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812077    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812080    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812083    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812086    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812089    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812092    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812095    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812098    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812101    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812104    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812106    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812109    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812112    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812114    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:10.814446 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812117    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812119    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812122    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812125    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812127    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812129    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812132    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812134    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812137    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812140    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812142    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812145    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812147    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812150    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812152    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812155    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812158    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812161    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812163    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812166    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:10.814946 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812168    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812171    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812173    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812176    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812180    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812184    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812187    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812189    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812192    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812194    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812196    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812199    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812201    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812205    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812207    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812209    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812212    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812214    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812217    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:10.815532 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812219    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812222    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812225    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812228    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812230    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812232    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812235    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812237    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812240    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812242    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812245    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812247    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812250    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812252    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812255    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812257    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812260    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812262    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812265    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812338    2556 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812346    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 17:52:10.816021 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812352    2556 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812356    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812378    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812382    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812386    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812391    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812394    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812397    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812400    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812403    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812406    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812410    2556 flags.go:64] FLAG: --cgroup-root=""
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812412    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812415    2556 flags.go:64] FLAG: --client-ca-file=""
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812419    2556 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812422    2556 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812427    2556 flags.go:64] FLAG: --cluster-domain=""
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812430    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812433    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812436    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812439    2556 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812442    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812445    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 17:52:10.816553 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812448    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812452    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812455    2556 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812458    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812461    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812464    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812468    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812472    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812475    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812478    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812481    2556 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812484    2556 flags.go:64] FLAG: --enable-server="true"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812487    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812491    2556 flags.go:64] FLAG: --event-burst="100"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812494    2556 flags.go:64] FLAG: --event-qps="50"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812497    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812501    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812504    2556 flags.go:64] FLAG: --eviction-hard=""
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812508    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812511    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812514    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812517    2556 flags.go:64] FLAG: --eviction-soft=""
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812520    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812523    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812526    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 17:52:10.817099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812529    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812532    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812535    2556 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812538    2556 flags.go:64] FLAG: --feature-gates=""
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812547    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812550    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812553    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812556    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812559    2556 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812562    2556 flags.go:64] FLAG: --help="false"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812565    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-135-253.ec2.internal"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812569    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812571    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812574    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812577    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812581    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812583    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812586    2556 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812589    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812591    2556 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812594    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812597    2556 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812600    2556 flags.go:64] FLAG: --kube-reserved=""
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812603    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 17:52:10.817751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812605    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812608    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812611    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812616    2556 flags.go:64] FLAG: --lock-file=""
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812618    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812621    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812624    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812630    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812634    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812637    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812639    2556 flags.go:64] FLAG: --logging-format="text"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812642    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812645    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812648    2556 flags.go:64] FLAG: --manifest-url=""
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812651    2556 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812655    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812659    2556 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812663    2556 flags.go:64] FLAG: --max-pods="110"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812666    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812669    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812672    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812675    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812677    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812680    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812683    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 17:52:10.818342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812691    2556 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812694    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812697    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812700    2556 flags.go:64] FLAG: --pod-cidr=""
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812703    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812708    2556 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812711    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812714    2556 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812717    2556 flags.go:64] FLAG: --port="10250"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812719    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812723    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b4980788153bdd71"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812727    2556 flags.go:64] FLAG: --qos-reserved=""
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812730    2556 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812733    2556 flags.go:64] FLAG: --register-node="true"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812736    2556 flags.go:64] FLAG: --register-with-taints=""
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812744    2556 flags.go:64] FLAG: --registry-burst="10"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812747    2556 flags.go:64] FLAG: --registry-qps="5"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812750    2556 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812753    2556 flags.go:64] FLAG: --reserved-memory=""
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812756    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812759    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812762    2556 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812765    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812770    2556 flags.go:64] FLAG: --runonce="false"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812772    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 17:52:10.818956 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812775    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812778    2556 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812781    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812784    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812787    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812790    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812793    2556 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812796    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812798    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812801    2556 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812804    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812807    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812810    2556 flags.go:64] FLAG: --system-cgroups=""
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812813    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812818    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812820    2556 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812823    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812828    2556 flags.go:64] FLAG: --tls-min-version=""
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812832    2556 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812835    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812838    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812841    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812844    2556 flags.go:64] FLAG: --v="2"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812848    2556 flags.go:64] FLAG: --version="false"
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812852    2556 flags.go:64] FLAG: --vmodule=""
Jul 29 17:52:10.819633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812856    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.812860    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812978    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812982    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812985    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812990    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812993    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812995    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.812998    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813000    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813003    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813006    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813008    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813011    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813014    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813016    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813019    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813021    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813024    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813028    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:10.820510 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813031    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813034    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813036    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813040    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813044    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813049    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813052    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813055    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813058    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813061    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813064    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813066    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813069    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813072    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813075    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813077    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813080    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813084    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813086    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:10.821386 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813089    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813092    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813094    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813097    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813099    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813102    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813105    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813107    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813110    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813112    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813115    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813117    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813120    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813122    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813125    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813127    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813130    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813132    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813136    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813139    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:10.822197 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813141    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813144    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813146    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813149    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813151    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813154    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813157    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813160    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813162    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813165    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813168    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813171    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813174    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813192    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813196    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813199    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813202    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813205    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813208    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813210    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:10.823086 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813213    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813216    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813218    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813221    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813224    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813226    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813229    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813232    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813234    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813236    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813240    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813243    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813246    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813248    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813251    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813253    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813256    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813259    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813261    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813265    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:10.823889 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813267    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.813270    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.814201    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.822162    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.822185    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.822210    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.822219    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822309    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822318    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822324    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822331    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822337    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822342    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822346    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:10.824468 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822351    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822356    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822378    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822383    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822387    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822391    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822395    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822399    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822404    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822408    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822412    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822416    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822420    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822425    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822429    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822433    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822437    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822442    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822446    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822450    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:10.824944 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822455    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822459    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822465    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822471    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822475    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822479    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822483    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822488    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822495    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822500    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822506    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822510    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822515    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822519    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822523    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822527    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822531    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822536    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822540    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822544    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:10.825448 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822548    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822553    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822557    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822561    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822566    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822570    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822575    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822579    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822583    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822587    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822591    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822595    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822600    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822604    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822609    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822614    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822618    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822622    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822626    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822630    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:10.826187 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822634    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822639    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822643    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822647    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822651    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822655    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822659    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822663    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822667    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822671    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822675    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822682    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822687    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822691    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822696    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822700    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822704    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822708    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822712    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:10.826957 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822716    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822720    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822724    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822729    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822733    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822737    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822741    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822746    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822750    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822756    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822760    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822764    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822769    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.822778    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822983    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:10.827591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822992    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.822997    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823002    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823006    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823011    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823016    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823020    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823025    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823029    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823034    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823038    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823044    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823050    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823056    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823062    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823067    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823072    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823076    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823081    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:10.828034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823085    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823089    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823094    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823098    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823102    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823107    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823112    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823116    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823121    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823126    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823131    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823135    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823139    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823143    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823148    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823152    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823156    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823161    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823165    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823169    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:10.828565 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823173    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823177    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823182    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823186    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823190    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823194    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823198    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823204    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823210    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823215    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823219    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823224    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823228    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823232    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823236    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823240    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823244    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823248    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823253    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:10.829056 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823257    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823261    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823266    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823271    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823275    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823279    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823283    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823287    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823291    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823296    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823300    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823304    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823308    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823312    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823316    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823321    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823325    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823329    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823333    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823337    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:10.829605 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823341    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823346    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823350    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823354    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823376    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823380    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823385    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823388    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823392    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823396    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823400    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823404    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823408    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823413    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823417    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823422    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823426    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823429    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823434    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:10.830084 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:10.823438    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:10.830634 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.823447    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:52:10.830634 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.823604    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 17:52:10.830634 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.827569    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 17:52:10.830634 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.829402    2556 server.go:1019] "Starting client certificate rotation"
Jul 29 17:52:10.830634 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.829507    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 17:52:10.830634 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.829541    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 17:52:10.832389 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.832356    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 17:52:10.835416 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.835396    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 17:52:10.846750 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.846729    2556 log.go:25] "Validated CRI v1 runtime API"
Jul 29 17:52:10.850792 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.850776    2556 log.go:25] "Validated CRI v1 image API"
Jul 29 17:52:10.851564 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.851547    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 17:52:10.856623 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.856603    2556 fs.go:135] Filesystem UUIDs: map[7416a2d6-acf6-430c-b5a1-1747540f26cb:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e3916638-d724-4c54-997a-8dfc786e7e90:/dev/nvme0n1p3]
Jul 29 17:52:10.856681 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.856622    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 17:52:10.862699 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.862593    2556 manager.go:216] Machine: {Timestamp:2026-07-29 17:52:10.861206975 +0000 UTC m=+0.369590556 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101766 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec240ec215a49c574862951c38617daf SystemUUID:ec240ec2-15a4-9c57-4862-951c38617daf BootID:e75c3059-b8e0-4d3f-93c7-82ea1f14d6ff Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e6:3b:0f:52:d1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e6:3b:0f:52:d1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:91:44:a4:9a:45 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 17:52:10.862699 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.862694    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 17:52:10.862799 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.862774    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 17:52:10.862960 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.862945    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 17:52:10.866396 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.866357    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 17:52:10.866528 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.866399    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-253.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 17:52:10.867266 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.867257    2556 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 17:52:10.867296 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.867269    2556 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 17:52:10.867296 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.867277    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 17:52:10.868097 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.868087    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 17:52:10.870206 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.870191    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 17:52:10.874843 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.874829    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 17:52:10.874957 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.874948    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 17:52:10.874997 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.874979    2556 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 17:52:10.874997 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.874989    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 17:52:10.875058 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.875000    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 17:52:10.875058 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.875009    2556 kubelet.go:398] "Adding apiserver pod source"
Jul 29 17:52:10.875058 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.875017    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 17:52:10.875393 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.875382    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 17:52:10.875436 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.875400    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 17:52:10.879254 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.879239    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 17:52:10.879884 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.879866    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qg95d"
Jul 29 17:52:10.882653 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.882639    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 17:52:10.882696 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.882671    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 17:52:10.884726 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884712    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 17:52:10.884776 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884733    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 17:52:10.884776 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884744    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 17:52:10.884776 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884753    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 17:52:10.884776 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884764    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 17:52:10.884776 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884773    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 17:52:10.884955 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884785    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 17:52:10.884955 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884793    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 17:52:10.884955 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884803    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 17:52:10.884955 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884812    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 17:52:10.884955 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884838    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 17:52:10.884955 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.884847    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 17:52:10.885498 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.885487    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 17:52:10.885535 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.885499    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 17:52:10.886953 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:10.886924    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 17:52:10.887027 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:10.886979    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-253.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 17:52:10.887214 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.887201    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qg95d"
Jul 29 17:52:10.889978 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.889965    2556 server.go:1268] "Started kubelet"
Jul 29 17:52:10.890098 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.890069    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 17:52:10.890192 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.890073    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 17:52:10.890192 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.890151    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 17:52:10.890885 ip-10-0-135-253 systemd[1]: Started Kubernetes Kubelet.
Jul 29 17:52:10.891332 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.891320    2556 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 17:52:10.892096 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.892079    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 17:52:10.897099 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:10.897075    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 17:52:10.897759 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.897743    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-253.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 17:52:10.899387 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:10.895283    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-253.ec2.internal.18c6d41ec350c50d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-253.ec2.internal,UID:ip-10-0-135-253.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-253.ec2.internal,},FirstTimestamp:2026-07-29 17:52:10.889938189 +0000 UTC m=+0.398321769,LastTimestamp:2026-07-29 17:52:10.889938189 +0000 UTC m=+0.398321769,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-253.ec2.internal,}"
Jul 29 17:52:10.901829 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.901807    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 17:52:10.901933 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.901814    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 17:52:10.901933 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:10.901916    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-253.ec2.internal\" not found"
Jul 29 17:52:10.902606 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.902505    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 17:52:10.902606 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.902522    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 17:52:10.902606 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.902565    2556 factory.go:55] Registering systemd factory
Jul 29 17:52:10.902606 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.902606    2556 factory.go:223] Registration of the systemd container factory successfully
Jul 29 17:52:10.902826 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.902614    2556 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 17:52:10.902826 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.902624    2556 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 17:52:10.902826 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.902807    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 17:52:10.904251 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.904233    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:10.904461 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.904446    2556 factory.go:153] Registering CRI-O factory
Jul 29 17:52:10.904536 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.904463    2556 factory.go:223] Registration of the crio container factory successfully
Jul 29 17:52:10.904536 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.904506    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 17:52:10.904536 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.904527    2556 factory.go:103] Registering Raw factory
Jul 29 17:52:10.904536 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.904540    2556 manager.go:1196] Started watching for new ooms in manager
Jul 29 17:52:10.906958 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.904854    2556 manager.go:349] Starting recovery of all containers
Jul 29 17:52:10.907529 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:10.907507    2556 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-253.ec2.internal\" not found" node="ip-10-0-135-253.ec2.internal"
Jul 29 17:52:10.914712 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.914698    2556 manager.go:354] Recovery completed
Jul 29 17:52:10.919634 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.919620    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:10.921426 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.921408    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:10.921504 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.921441    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:10.921504 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.921455    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:10.921916 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.921902    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 17:52:10.921916 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.921911    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:10.921916 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.921913    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 17:52:10.922061 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.921928    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 17:52:10.922540 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.922525    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:10.922591 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.922554    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:10.922591 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:10.922568    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:11.002680 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.002655    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-253.ec2.internal\" not found"
Jul 29 17:52:11.068114 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.068052    2556 policy_none.go:49] "None policy: Start"
Jul 29 17:52:11.068114 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.068077    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 17:52:11.068114 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.068089    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 17:52:11.070221 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.070200    2556 policy_none.go:47] "Start"
Jul 29 17:52:11.071167 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.071147    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 17:52:11.072538 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.072519    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 17:52:11.072624 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.072562    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 17:52:11.072624 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.072590    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 17:52:11.072699 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.072630    2556 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Jul 29 17:52:11.075472 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.075313    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:11.096865 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.096853    2556 manager.go:337] "Starting Device Plugin manager"
Jul 29 17:52:11.096944 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.096890    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 17:52:11.096944 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.096902    2556 server.go:87] "Starting device plugin registration server"
Jul 29 17:52:11.097179 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.097169    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 17:52:11.097213 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.097181    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 17:52:11.097310 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.097287    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 17:52:11.097391 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.097384    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 17:52:11.097447 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.097393    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 17:52:11.097898 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.097884    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 17:52:11.097958 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.097927    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-253.ec2.internal\" not found"
Jul 29 17:52:11.173596 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.173539    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal"]
Jul 29 17:52:11.173663 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.173640    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:11.174475 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.174460    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:11.174539 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.174485    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:11.174539 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.174496    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:11.176876 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.176859    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:11.176987 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.176974    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.177022 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.177005    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:11.177563 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.177548    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:11.177563 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.177556    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:11.177696 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.177572    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:11.177696 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.177579    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:11.177696 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.177594    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:11.177696 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.177581    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:11.179793 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.179781    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.179851 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.179807    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:11.181486 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.181469    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:11.181556 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.181500    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:11.181556 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.181517    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:11.197338 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.197321    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:11.198018 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.198006    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:11.198074 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.198028    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:11.198074 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.198038    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:11.198074 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.198056    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.203155 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.203140    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b348111e1b2cbede8ff3b7ecfdb4a2f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal\" (UID: \"2b348111e1b2cbede8ff3b7ecfdb4a2f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.203208 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.203162    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9a47ae04fab76d00c2743727b114e18-config\") pod \"kube-apiserver-proxy-ip-10-0-135-253.ec2.internal\" (UID: \"e9a47ae04fab76d00c2743727b114e18\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.203208 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.203179    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b348111e1b2cbede8ff3b7ecfdb4a2f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal\" (UID: \"2b348111e1b2cbede8ff3b7ecfdb4a2f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.206009 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.205997    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.206058 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.206014    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-253.ec2.internal\": node \"ip-10-0-135-253.ec2.internal\" not found"
Jul 29 17:52:11.206894 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.206881    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-253.ec2.internal\" not found" node="ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.211432 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.211417    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-253.ec2.internal\" not found" node="ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.219055 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.219037    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-253.ec2.internal\" not found"
Jul 29 17:52:11.303731 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.303705    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9a47ae04fab76d00c2743727b114e18-config\") pod \"kube-apiserver-proxy-ip-10-0-135-253.ec2.internal\" (UID: \"e9a47ae04fab76d00c2743727b114e18\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.303731 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.303732    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b348111e1b2cbede8ff3b7ecfdb4a2f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal\" (UID: \"2b348111e1b2cbede8ff3b7ecfdb4a2f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.303867 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.303748    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b348111e1b2cbede8ff3b7ecfdb4a2f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal\" (UID: \"2b348111e1b2cbede8ff3b7ecfdb4a2f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.303867 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.303803    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b348111e1b2cbede8ff3b7ecfdb4a2f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal\" (UID: \"2b348111e1b2cbede8ff3b7ecfdb4a2f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.303867 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.303804    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e9a47ae04fab76d00c2743727b114e18-config\") pod \"kube-apiserver-proxy-ip-10-0-135-253.ec2.internal\" (UID: \"e9a47ae04fab76d00c2743727b114e18\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.303867 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.303827    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b348111e1b2cbede8ff3b7ecfdb4a2f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal\" (UID: \"2b348111e1b2cbede8ff3b7ecfdb4a2f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.319987 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.319939    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-253.ec2.internal\" not found"
Jul 29 17:52:11.420695 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.420665    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-253.ec2.internal\" not found"
Jul 29 17:52:11.509163 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.509137    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.515201 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.515185    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.520804 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.520786    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-253.ec2.internal\" not found"
Jul 29 17:52:11.621284 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.621200    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-253.ec2.internal\" not found"
Jul 29 17:52:11.721576 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.721536    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-253.ec2.internal\" not found"
Jul 29 17:52:11.764993 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.764968    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:11.803156 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.803133    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.820041 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.820020    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 17:52:11.821890 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.821872    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.829423 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.829407    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 17:52:11.829564 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.829536    2556 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a80d6ce4610e84eb2bc106483ed5069b-7bfb7e5c08df3b58.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.135.253:38244->3.223.187.143:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal"
Jul 29 17:52:11.829632 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.829571    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 17:52:11.829632 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.829574    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 17:52:11.829632 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.829578    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 17:52:11.877784 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.877769    2556 apiserver.go:52] "Watching apiserver"
Jul 29 17:52:11.896646 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.896611    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 17:47:11 +0000 UTC" deadline="2028-02-26 10:45:54.749227961 +0000 UTC"
Jul 29 17:52:11.896646 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.896640    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13840h53m42.85259145s"
Jul 29 17:52:11.902991 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.902975    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 17:52:11.905273 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.905249    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 17:52:11.906491 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.906471    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-5qf55","kube-system/konnectivity-agent-2sqk8","kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l","openshift-cluster-node-tuning-operator/tuned-p69dr","openshift-dns/node-resolver-jf97c","openshift-image-registry/node-ca-cttw7","openshift-multus/multus-7hdr8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal","openshift-multus/multus-additional-cni-plugins-wkw8q","openshift-multus/network-metrics-daemon-vcnql","openshift-network-diagnostics/network-check-target-pv5rx","openshift-network-operator/iptables-alerter-52z29","openshift-ovn-kubernetes/ovnkube-node-t9wvx"]
Jul 29 17:52:11.909170 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.909150    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:11.909260 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.909234    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:11.911451 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.911436    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:11.913761 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.913737    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:11.916049 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.916032    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:11.918314 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.918295    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jf97c"
Jul 29 17:52:11.921335 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.921304    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cttw7"
Jul 29 17:52:11.922541 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.922509    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 17:52:11.922541 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.922534    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:11.922739 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.922546    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 17:52:11.922739 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.922517    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 17:52:11.922739 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.922558    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 17:52:11.922871 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.922779    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 17:52:11.923749 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.923732    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:11.924124 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.924097    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 17:52:11.924230 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.924213    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7lc9b\""
Jul 29 17:52:11.924323 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.924286    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tbtcr\""
Jul 29 17:52:11.924323 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.924300    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nhrjs\""
Jul 29 17:52:11.924458 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.924341    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9w5bh\""
Jul 29 17:52:11.924458 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.924396    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 17:52:11.924828 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.924811    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 17:52:11.925373 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.925359    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 17:52:11.925813 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.925798    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 17:52:11.925873 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.925846    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 17:52:11.926055 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.926038    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wjjxx\""
Jul 29 17:52:11.926136 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.926120    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:11.926529 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.926513    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 17:52:11.929272 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.929253    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 17:52:11.929527 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.929509    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:11.929606 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.929584    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:11.930601 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.930584    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 17:52:11.931656 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.931639    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:11.931745 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:11.931682    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:11.934070 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.934051    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-52z29"
Jul 29 17:52:11.934993 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.934977    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 17:52:11.935673 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.935659    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 17:52:11.935821 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.935808    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 17:52:11.935976 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.935962    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 17:52:11.936158 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.936144    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-djbq2\""
Jul 29 17:52:11.936296 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.936284    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qgz2n\""
Jul 29 17:52:11.936404 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.936387    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:11.937176 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.937160    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:11.937232 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.937216    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-plb7c\""
Jul 29 17:52:11.937359 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.937345    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 17:52:11.940323 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.940306    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 17:52:11.940502 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.940487    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 17:52:11.940585 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.940562    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 17:52:11.940901 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.940886    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 17:52:11.940967 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.940952    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8svpb\""
Jul 29 17:52:11.941263 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.941247    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 17:52:11.942309 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.942296    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 17:52:11.942620 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.942606    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 17:52:11.951933 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.951915    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w2t65"
Jul 29 17:52:11.968113 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:11.968095    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w2t65"
Jul 29 17:52:12.005038 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.004986    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 17:52:12.007811 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.007790    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6fa20546-4be3-4fe7-ba04-6428acad4a32-ovnkube-config\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.007905 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.007816    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.007905 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.007834    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-registration-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.007905 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.007852    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6adbb4b3-7133-49d5-844b-182ed5602391-cni-binary-copy\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.007905 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.007877    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6adbb4b3-7133-49d5-844b-182ed5602391-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.008060 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.007908    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-run-netns\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.008060 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.007936    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-var-lib-cni-multus\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.008060 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.007970    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-run-ovn\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.008060 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.007988    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.008060 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008003    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f6cf20b-051e-4536-af42-e026c961f43b-tmp\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.008060 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008018    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-system-cni-dir\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.008060 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008033    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-slash\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.008060 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008048    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-var-lib-openvswitch\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008083    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-cni-netd\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008118    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/026c3a54-f3f1-4f0f-88d6-681bd039200d-konnectivity-ca\") pod \"konnectivity-agent-2sqk8\" (UID: \"026c3a54-f3f1-4f0f-88d6-681bd039200d\") " pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008152    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-host\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008179    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008196    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjvj9\" (UniqueName: \"kubernetes.io/projected/6adbb4b3-7133-49d5-844b-182ed5602391-kube-api-access-cjvj9\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008214    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-socket-dir-parent\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008230    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008245    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-node-log\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008266    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-etc-selinux\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008283    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8zj8\" (UniqueName: \"kubernetes.io/projected/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-kube-api-access-q8zj8\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008298    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-var-lib-kubelet\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.008328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008322    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-run-multus-certs\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008341    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76ae3d15-ef4e-4868-abdd-c86e08c8886c-iptables-alerter-script\") pod \"iptables-alerter-52z29\" (UID: \"76ae3d15-ef4e-4868-abdd-c86e08c8886c\") " pod="openshift-network-operator/iptables-alerter-52z29"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008358    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-run-systemd\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008384    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fa20546-4be3-4fe7-ba04-6428acad4a32-ovn-node-metrics-cert\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008406    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-conf-dir\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008422    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-kubernetes\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008449    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6adbb4b3-7133-49d5-844b-182ed5602391-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008465    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-cni-dir\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008481    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-hostroot\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008501    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-etc-openvswitch\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008517    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/026c3a54-f3f1-4f0f-88d6-681bd039200d-agent-certs\") pod \"konnectivity-agent-2sqk8\" (UID: \"026c3a54-f3f1-4f0f-88d6-681bd039200d\") " pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008532    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-sys-fs\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008545    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-modprobe-d\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008586    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-cnibin\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008620    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfqmp\" (UniqueName: \"kubernetes.io/projected/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-kube-api-access-rfqmp\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008655    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.008947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008682    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmsdt\" (UniqueName: \"kubernetes.io/projected/6fa20546-4be3-4fe7-ba04-6428acad4a32-kube-api-access-pmsdt\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008709    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-run\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008733    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-os-release\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008756    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zvsj\" (UniqueName: \"kubernetes.io/projected/ad618126-a7f9-40d0-a92e-4b80729d8384-kube-api-access-2zvsj\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008782    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9a962e36-54ac-40e3-8c51-c317c81f2133-kubelet-config\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008808    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-socket-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008832    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-cnibin\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008855    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-log-socket\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008878    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-sys\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008900    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlg5s\" (UniqueName: \"kubernetes.io/projected/eb4927fd-bc0a-4316-b43f-fb9144018eab-kube-api-access-rlg5s\") pod \"node-ca-cttw7\" (UID: \"eb4927fd-bc0a-4316-b43f-fb9144018eab\") " pod="openshift-image-registry/node-ca-cttw7"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008924    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-etc-kubernetes\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008947    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49jnc\" (UniqueName: \"kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc\") pod \"network-check-target-pv5rx\" (UID: \"ced120c9-89f5-45bb-85b8-37d7e0b82f9e\") " pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008970    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-systemd\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.008995    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76ae3d15-ef4e-4868-abdd-c86e08c8886c-host-slash\") pod \"iptables-alerter-52z29\" (UID: \"76ae3d15-ef4e-4868-abdd-c86e08c8886c\") " pod="openshift-network-operator/iptables-alerter-52z29"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009020    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009044    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-run-k8s-cni-cncf-io\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.009769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009068    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-var-lib-cni-bin\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009091    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-kubelet\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009114    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9a962e36-54ac-40e3-8c51-c317c81f2133-dbus\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009140    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-lib-modules\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009167    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-os-release\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009191    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6fa20546-4be3-4fe7-ba04-6428acad4a32-env-overrides\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009215    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-sysctl-d\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009247    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-var-lib-kubelet\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009278    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3927a342-2b8b-47bd-bbb3-f78bbce3802a-hosts-file\") pod \"node-resolver-jf97c\" (UID: \"3927a342-2b8b-47bd-bbb3-f78bbce3802a\") " pod="openshift-dns/node-resolver-jf97c"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009302    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eb4927fd-bc0a-4316-b43f-fb9144018eab-serviceca\") pod \"node-ca-cttw7\" (UID: \"eb4927fd-bc0a-4316-b43f-fb9144018eab\") " pod="openshift-image-registry/node-ca-cttw7"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009325    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-sysconfig\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009358    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ft7m\" (UniqueName: \"kubernetes.io/projected/3927a342-2b8b-47bd-bbb3-f78bbce3802a-kube-api-access-2ft7m\") pod \"node-resolver-jf97c\" (UID: \"3927a342-2b8b-47bd-bbb3-f78bbce3802a\") " pod="openshift-dns/node-resolver-jf97c"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009383    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eb4927fd-bc0a-4316-b43f-fb9144018eab-host\") pod \"node-ca-cttw7\" (UID: \"eb4927fd-bc0a-4316-b43f-fb9144018eab\") " pod="openshift-image-registry/node-ca-cttw7"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009408    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-cni-binary-copy\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009451    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77v4x\" (UniqueName: \"kubernetes.io/projected/76ae3d15-ef4e-4868-abdd-c86e08c8886c-kube-api-access-77v4x\") pod \"iptables-alerter-52z29\" (UID: \"76ae3d15-ef4e-4868-abdd-c86e08c8886c\") " pod="openshift-network-operator/iptables-alerter-52z29"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009477    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-cni-bin\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009500    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6fa20546-4be3-4fe7-ba04-6428acad4a32-ovnkube-script-lib\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.010345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009535    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-sysctl-conf\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.010951 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009560    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-systemd-units\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.010951 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009584    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-run-openvswitch\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.010951 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009640    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-device-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.010951 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009666    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7f6cf20b-051e-4536-af42-e026c961f43b-etc-tuned\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.010951 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009692    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnv58\" (UniqueName: \"kubernetes.io/projected/7f6cf20b-051e-4536-af42-e026c961f43b-kube-api-access-jnv58\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.010951 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009716    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3927a342-2b8b-47bd-bbb3-f78bbce3802a-tmp-dir\") pod \"node-resolver-jf97c\" (UID: \"3927a342-2b8b-47bd-bbb3-f78bbce3802a\") " pod="openshift-dns/node-resolver-jf97c"
Jul 29 17:52:12.010951 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009741    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-system-cni-dir\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.010951 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009769    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-daemon-config\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.010951 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.009824    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-run-netns\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.024545 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.024525    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:12.110989 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.110957    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-run-multus-certs\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.111149 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.110998    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76ae3d15-ef4e-4868-abdd-c86e08c8886c-iptables-alerter-script\") pod \"iptables-alerter-52z29\" (UID: \"76ae3d15-ef4e-4868-abdd-c86e08c8886c\") " pod="openshift-network-operator/iptables-alerter-52z29"
Jul 29 17:52:12.111149 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111023    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-run-systemd\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.111149 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111046    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fa20546-4be3-4fe7-ba04-6428acad4a32-ovn-node-metrics-cert\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.111149 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111065    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-run-multus-certs\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.111149 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111068    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-conf-dir\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.111149 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111112    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-kubernetes\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.111149 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111120    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-run-systemd\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.111149 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111131    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6adbb4b3-7133-49d5-844b-182ed5602391-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111159    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-cni-dir\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111184    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-hostroot\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111209    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-etc-openvswitch\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111238    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/026c3a54-f3f1-4f0f-88d6-681bd039200d-agent-certs\") pod \"konnectivity-agent-2sqk8\" (UID: \"026c3a54-f3f1-4f0f-88d6-681bd039200d\") " pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111255    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-kubernetes\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111265    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-sys-fs\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111282    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-hostroot\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111301    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-modprobe-d\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111323    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-sys-fs\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111330    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-cnibin\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111336    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-conf-dir\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111357    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfqmp\" (UniqueName: \"kubernetes.io/projected/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-kube-api-access-rfqmp\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111377    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-etc-openvswitch\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111403    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111431    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-cni-dir\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111433    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmsdt\" (UniqueName: \"kubernetes.io/projected/6fa20546-4be3-4fe7-ba04-6428acad4a32-kube-api-access-pmsdt\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111472    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-modprobe-d\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.111607 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111478    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-run\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111507    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-os-release\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111516    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-cnibin\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111536    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zvsj\" (UniqueName: \"kubernetes.io/projected/ad618126-a7f9-40d0-a92e-4b80729d8384-kube-api-access-2zvsj\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111562    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9a962e36-54ac-40e3-8c51-c317c81f2133-kubelet-config\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111563    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111468    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111597    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-socket-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111624    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-cnibin\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111630    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-os-release\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111660    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-log-socket\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111686    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-sys\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111693    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76ae3d15-ef4e-4868-abdd-c86e08c8886c-iptables-alerter-script\") pod \"iptables-alerter-52z29\" (UID: \"76ae3d15-ef4e-4868-abdd-c86e08c8886c\") " pod="openshift-network-operator/iptables-alerter-52z29"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111707    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-cnibin\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111714    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlg5s\" (UniqueName: \"kubernetes.io/projected/eb4927fd-bc0a-4316-b43f-fb9144018eab-kube-api-access-rlg5s\") pod \"node-ca-cttw7\" (UID: \"eb4927fd-bc0a-4316-b43f-fb9144018eab\") " pod="openshift-image-registry/node-ca-cttw7"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111743    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6adbb4b3-7133-49d5-844b-182ed5602391-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111752    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-run\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111756    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-etc-kubernetes\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.112388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111791    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-etc-kubernetes\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111801    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9a962e36-54ac-40e3-8c51-c317c81f2133-kubelet-config\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111825    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jnc\" (UniqueName: \"kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc\") pod \"network-check-target-pv5rx\" (UID: \"ced120c9-89f5-45bb-85b8-37d7e0b82f9e\") " pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111827    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-socket-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111855    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-systemd\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111948    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76ae3d15-ef4e-4868-abdd-c86e08c8886c-host-slash\") pod \"iptables-alerter-52z29\" (UID: \"76ae3d15-ef4e-4868-abdd-c86e08c8886c\") " pod="openshift-network-operator/iptables-alerter-52z29"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111977    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112006    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-run-k8s-cni-cncf-io\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112025    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-log-socket\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112033    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-var-lib-cni-bin\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112059    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76ae3d15-ef4e-4868-abdd-c86e08c8886c-host-slash\") pod \"iptables-alerter-52z29\" (UID: \"76ae3d15-ef4e-4868-abdd-c86e08c8886c\") " pod="openshift-network-operator/iptables-alerter-52z29"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112066    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-kubelet\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112101    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-kubelet\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112101    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-var-lib-cni-bin\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112116    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9a962e36-54ac-40e3-8c51-c317c81f2133-dbus\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.111859    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-sys\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112139    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-run-k8s-cni-cncf-io\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112154    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-lib-modules\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.113212 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.112159    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112181    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-os-release\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112183    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9a962e36-54ac-40e3-8c51-c317c81f2133-dbus\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112207    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6fa20546-4be3-4fe7-ba04-6428acad4a32-env-overrides\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112210    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-systemd\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112258    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-os-release\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.112266    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret podName:9a962e36-54ac-40e3-8c51-c317c81f2133 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:12.612218566 +0000 UTC m=+2.120602150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret") pod "global-pull-secret-syncer-5qf55" (UID: "9a962e36-54ac-40e3-8c51-c317c81f2133") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112279    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-lib-modules\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112292    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-sysctl-d\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112326    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-var-lib-kubelet\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112354    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3927a342-2b8b-47bd-bbb3-f78bbce3802a-hosts-file\") pod \"node-resolver-jf97c\" (UID: \"3927a342-2b8b-47bd-bbb3-f78bbce3802a\") " pod="openshift-dns/node-resolver-jf97c"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112378    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eb4927fd-bc0a-4316-b43f-fb9144018eab-serviceca\") pod \"node-ca-cttw7\" (UID: \"eb4927fd-bc0a-4316-b43f-fb9144018eab\") " pod="openshift-image-registry/node-ca-cttw7"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112414    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-var-lib-kubelet\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112418    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-sysconfig\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112422    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3927a342-2b8b-47bd-bbb3-f78bbce3802a-hosts-file\") pod \"node-resolver-jf97c\" (UID: \"3927a342-2b8b-47bd-bbb3-f78bbce3802a\") " pod="openshift-dns/node-resolver-jf97c"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112475    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-sysconfig\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112514    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ft7m\" (UniqueName: \"kubernetes.io/projected/3927a342-2b8b-47bd-bbb3-f78bbce3802a-kube-api-access-2ft7m\") pod \"node-resolver-jf97c\" (UID: \"3927a342-2b8b-47bd-bbb3-f78bbce3802a\") " pod="openshift-dns/node-resolver-jf97c"
Jul 29 17:52:12.114242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112536    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-sysctl-d\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112547    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eb4927fd-bc0a-4316-b43f-fb9144018eab-host\") pod \"node-ca-cttw7\" (UID: \"eb4927fd-bc0a-4316-b43f-fb9144018eab\") " pod="openshift-image-registry/node-ca-cttw7"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112586    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eb4927fd-bc0a-4316-b43f-fb9144018eab-host\") pod \"node-ca-cttw7\" (UID: \"eb4927fd-bc0a-4316-b43f-fb9144018eab\") " pod="openshift-image-registry/node-ca-cttw7"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112710    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-cni-binary-copy\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112747    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6fa20546-4be3-4fe7-ba04-6428acad4a32-env-overrides\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112750    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77v4x\" (UniqueName: \"kubernetes.io/projected/76ae3d15-ef4e-4868-abdd-c86e08c8886c-kube-api-access-77v4x\") pod \"iptables-alerter-52z29\" (UID: \"76ae3d15-ef4e-4868-abdd-c86e08c8886c\") " pod="openshift-network-operator/iptables-alerter-52z29"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112791    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-cni-bin\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112817    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6fa20546-4be3-4fe7-ba04-6428acad4a32-ovnkube-script-lib\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112845    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-sysctl-conf\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112872    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-systemd-units\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112889    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-cni-bin\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112902    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-run-openvswitch\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.112951    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-systemd-units\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113017    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-etc-sysctl-conf\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113015    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-device-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113055    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-run-openvswitch\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113064    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-device-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113058    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7f6cf20b-051e-4536-af42-e026c961f43b-etc-tuned\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.115100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113129    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnv58\" (UniqueName: \"kubernetes.io/projected/7f6cf20b-051e-4536-af42-e026c961f43b-kube-api-access-jnv58\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113157    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3927a342-2b8b-47bd-bbb3-f78bbce3802a-tmp-dir\") pod \"node-resolver-jf97c\" (UID: \"3927a342-2b8b-47bd-bbb3-f78bbce3802a\") " pod="openshift-dns/node-resolver-jf97c"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113184    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-system-cni-dir\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113211    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-daemon-config\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113235    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-run-netns\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113269    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6fa20546-4be3-4fe7-ba04-6428acad4a32-ovnkube-config\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113298    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113323    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-registration-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113349    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6adbb4b3-7133-49d5-844b-182ed5602391-cni-binary-copy\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113377    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6adbb4b3-7133-49d5-844b-182ed5602391-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113388    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eb4927fd-bc0a-4316-b43f-fb9144018eab-serviceca\") pod \"node-ca-cttw7\" (UID: \"eb4927fd-bc0a-4316-b43f-fb9144018eab\") " pod="openshift-image-registry/node-ca-cttw7"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113415    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-run-netns\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113466    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-cni-binary-copy\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113482    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-run-netns\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113482    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-run-netns\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113507    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-var-lib-cni-multus\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113511    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6fa20546-4be3-4fe7-ba04-6428acad4a32-ovnkube-script-lib\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.115981 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113583    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-system-cni-dir\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113587    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113628    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-var-lib-cni-multus\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113848    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-run-ovn\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113881    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113936    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f6cf20b-051e-4536-af42-e026c961f43b-tmp\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113963    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-system-cni-dir\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113990    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-slash\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114012    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6adbb4b3-7133-49d5-844b-182ed5602391-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114053    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-var-lib-openvswitch\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114069    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-system-cni-dir\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114080    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-cni-netd\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114110    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/026c3a54-f3f1-4f0f-88d6-681bd039200d-konnectivity-ca\") pod \"konnectivity-agent-2sqk8\" (UID: \"026c3a54-f3f1-4f0f-88d6-681bd039200d\") " pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114113    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-registration-dir\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114118    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-daemon-config\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114142    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-host\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114155    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-slash\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.116805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114172    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114191    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114201    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjvj9\" (UniqueName: \"kubernetes.io/projected/6adbb4b3-7133-49d5-844b-182ed5602391-kube-api-access-cjvj9\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114250    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-socket-dir-parent\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114278    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114290    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f6cf20b-051e-4536-af42-e026c961f43b-host\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114304    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-node-log\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114330    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-etc-selinux\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114358    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8zj8\" (UniqueName: \"kubernetes.io/projected/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-kube-api-access-q8zj8\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114253    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-run-ovn\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.113883    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3927a342-2b8b-47bd-bbb3-f78bbce3802a-tmp-dir\") pod \"node-resolver-jf97c\" (UID: \"3927a342-2b8b-47bd-bbb3-f78bbce3802a\") " pod="openshift-dns/node-resolver-jf97c"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114468    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-var-lib-kubelet\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114519    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-host-cni-netd\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114544    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-host-var-lib-kubelet\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114594    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-multus-socket-dir-parent\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114612    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6fa20546-4be3-4fe7-ba04-6428acad4a32-ovnkube-config\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.114664    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:12.117322 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.114760    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs podName:ad618126-a7f9-40d0-a92e-4b80729d8384 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:12.614740074 +0000 UTC m=+2.123123657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs") pod "network-metrics-daemon-vcnql" (UID: "ad618126-a7f9-40d0-a92e-4b80729d8384") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:12.117855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114758    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6adbb4b3-7133-49d5-844b-182ed5602391-cni-binary-copy\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.117855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114788    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6adbb4b3-7133-49d5-844b-182ed5602391-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.117855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114802    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-node-log\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.117855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.114927    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fa20546-4be3-4fe7-ba04-6428acad4a32-var-lib-openvswitch\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.117855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.115020    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-etc-selinux\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.117855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.115830    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/026c3a54-f3f1-4f0f-88d6-681bd039200d-konnectivity-ca\") pod \"konnectivity-agent-2sqk8\" (UID: \"026c3a54-f3f1-4f0f-88d6-681bd039200d\") " pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:12.117855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.115998    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fa20546-4be3-4fe7-ba04-6428acad4a32-ovn-node-metrics-cert\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.117855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.116113    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/026c3a54-f3f1-4f0f-88d6-681bd039200d-agent-certs\") pod \"konnectivity-agent-2sqk8\" (UID: \"026c3a54-f3f1-4f0f-88d6-681bd039200d\") " pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:12.117855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.116916    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7f6cf20b-051e-4536-af42-e026c961f43b-etc-tuned\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.117855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.117109    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f6cf20b-051e-4536-af42-e026c961f43b-tmp\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.122826 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.122807    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:12.122826 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.122828    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:12.122996 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.122840    2556 projected.go:196] Error preparing data for projected volume kube-api-access-49jnc for pod openshift-network-diagnostics/network-check-target-pv5rx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:12.122996 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.122898    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc podName:ced120c9-89f5-45bb-85b8-37d7e0b82f9e nodeName:}" failed. No retries permitted until 2026-07-29 17:52:12.622880056 +0000 UTC m=+2.131263625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-49jnc" (UniqueName: "kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc") pod "network-check-target-pv5rx" (UID: "ced120c9-89f5-45bb-85b8-37d7e0b82f9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:12.124794 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.124769    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfqmp\" (UniqueName: \"kubernetes.io/projected/7c9c6ced-e28d-42f1-88a1-9a06ec53ba62-kube-api-access-rfqmp\") pod \"multus-7hdr8\" (UID: \"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62\") " pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.125223 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.125187    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmsdt\" (UniqueName: \"kubernetes.io/projected/6fa20546-4be3-4fe7-ba04-6428acad4a32-kube-api-access-pmsdt\") pod \"ovnkube-node-t9wvx\" (UID: \"6fa20546-4be3-4fe7-ba04-6428acad4a32\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.126087 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.126067    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8zj8\" (UniqueName: \"kubernetes.io/projected/9e08cf6d-6e08-4ebb-8c6c-460dca8e8117-kube-api-access-q8zj8\") pod \"aws-ebs-csi-driver-node-9tl8l\" (UID: \"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.126231 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.126206    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ft7m\" (UniqueName: \"kubernetes.io/projected/3927a342-2b8b-47bd-bbb3-f78bbce3802a-kube-api-access-2ft7m\") pod \"node-resolver-jf97c\" (UID: \"3927a342-2b8b-47bd-bbb3-f78bbce3802a\") " pod="openshift-dns/node-resolver-jf97c"
Jul 29 17:52:12.126540 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.126495    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlg5s\" (UniqueName: \"kubernetes.io/projected/eb4927fd-bc0a-4316-b43f-fb9144018eab-kube-api-access-rlg5s\") pod \"node-ca-cttw7\" (UID: \"eb4927fd-bc0a-4316-b43f-fb9144018eab\") " pod="openshift-image-registry/node-ca-cttw7"
Jul 29 17:52:12.126618 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.126567    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zvsj\" (UniqueName: \"kubernetes.io/projected/ad618126-a7f9-40d0-a92e-4b80729d8384-kube-api-access-2zvsj\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:12.126618 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.126605    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjvj9\" (UniqueName: \"kubernetes.io/projected/6adbb4b3-7133-49d5-844b-182ed5602391-kube-api-access-cjvj9\") pod \"multus-additional-cni-plugins-wkw8q\" (UID: \"6adbb4b3-7133-49d5-844b-182ed5602391\") " pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.130256 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.130204    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnv58\" (UniqueName: \"kubernetes.io/projected/7f6cf20b-051e-4536-af42-e026c961f43b-kube-api-access-jnv58\") pod \"tuned-p69dr\" (UID: \"7f6cf20b-051e-4536-af42-e026c961f43b\") " pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.130739 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.130722    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77v4x\" (UniqueName: \"kubernetes.io/projected/76ae3d15-ef4e-4868-abdd-c86e08c8886c-kube-api-access-77v4x\") pod \"iptables-alerter-52z29\" (UID: \"76ae3d15-ef4e-4868-abdd-c86e08c8886c\") " pod="openshift-network-operator/iptables-alerter-52z29"
Jul 29 17:52:12.133260 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:12.133236    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b348111e1b2cbede8ff3b7ecfdb4a2f.slice/crio-50e3aa52896f30080aa7ea76330e650f0cf283048a7d3c38700abcd1f26a2570 WatchSource:0}: Error finding container 50e3aa52896f30080aa7ea76330e650f0cf283048a7d3c38700abcd1f26a2570: Status 404 returned error can't find the container with id 50e3aa52896f30080aa7ea76330e650f0cf283048a7d3c38700abcd1f26a2570
Jul 29 17:52:12.137725 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:12.137700    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9a47ae04fab76d00c2743727b114e18.slice/crio-866c3b6b0639faf0b0b2ba3068a83b05cd167a8767c80e26b0ee5c19800c1720 WatchSource:0}: Error finding container 866c3b6b0639faf0b0b2ba3068a83b05cd167a8767c80e26b0ee5c19800c1720: Status 404 returned error can't find the container with id 866c3b6b0639faf0b0b2ba3068a83b05cd167a8767c80e26b0ee5c19800c1720
Jul 29 17:52:12.140682 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.140660    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 17:52:12.261583 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.261533    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:12.266967 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.266950    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l"
Jul 29 17:52:12.272045 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:12.272019    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026c3a54_f3f1_4f0f_88d6_681bd039200d.slice/crio-9d5e5b23c0a272115f3949ffda940fc756bf4f2feec7889da1635278816a9130 WatchSource:0}: Error finding container 9d5e5b23c0a272115f3949ffda940fc756bf4f2feec7889da1635278816a9130: Status 404 returned error can't find the container with id 9d5e5b23c0a272115f3949ffda940fc756bf4f2feec7889da1635278816a9130
Jul 29 17:52:12.275963 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:12.275937    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e08cf6d_6e08_4ebb_8c6c_460dca8e8117.slice/crio-3d8bc0fc045421100e3e8041c1d6141dedf0b773fc2c7b247f62030758bfa8b4 WatchSource:0}: Error finding container 3d8bc0fc045421100e3e8041c1d6141dedf0b773fc2c7b247f62030758bfa8b4: Status 404 returned error can't find the container with id 3d8bc0fc045421100e3e8041c1d6141dedf0b773fc2c7b247f62030758bfa8b4
Jul 29 17:52:12.284732 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.284712    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p69dr"
Jul 29 17:52:12.287042 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.287024    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jf97c"
Jul 29 17:52:12.295140 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:12.295116    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6cf20b_051e_4536_af42_e026c961f43b.slice/crio-347c9139979d5e0d295eb10814ae9fdec277e0e5686978eb39ab980a8876586b WatchSource:0}: Error finding container 347c9139979d5e0d295eb10814ae9fdec277e0e5686978eb39ab980a8876586b: Status 404 returned error can't find the container with id 347c9139979d5e0d295eb10814ae9fdec277e0e5686978eb39ab980a8876586b
Jul 29 17:52:12.298034 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:12.298004    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3927a342_2b8b_47bd_bbb3_f78bbce3802a.slice/crio-3ff4dee80ccccaa8734e6b93a8e695c7822487d64a02b094d02af252a51bb967 WatchSource:0}: Error finding container 3ff4dee80ccccaa8734e6b93a8e695c7822487d64a02b094d02af252a51bb967: Status 404 returned error can't find the container with id 3ff4dee80ccccaa8734e6b93a8e695c7822487d64a02b094d02af252a51bb967
Jul 29 17:52:12.315254 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.315236    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cttw7"
Jul 29 17:52:12.324431 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:12.324412    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4927fd_bc0a_4316_b43f_fb9144018eab.slice/crio-bc691c142afe1d0d0629ad80d5f9cfce4333c7962ff0f61836ed2b7bc8fe7cf5 WatchSource:0}: Error finding container bc691c142afe1d0d0629ad80d5f9cfce4333c7962ff0f61836ed2b7bc8fe7cf5: Status 404 returned error can't find the container with id bc691c142afe1d0d0629ad80d5f9cfce4333c7962ff0f61836ed2b7bc8fe7cf5
Jul 29 17:52:12.333056 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.333040    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7hdr8"
Jul 29 17:52:12.341027 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:12.341000    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9c6ced_e28d_42f1_88a1_9a06ec53ba62.slice/crio-f1453035101ba2fe5ef2427148581051e2dcd5fa0e19cf2b20af35072564f6a3 WatchSource:0}: Error finding container f1453035101ba2fe5ef2427148581051e2dcd5fa0e19cf2b20af35072564f6a3: Status 404 returned error can't find the container with id f1453035101ba2fe5ef2427148581051e2dcd5fa0e19cf2b20af35072564f6a3
Jul 29 17:52:12.347890 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.347871    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wkw8q"
Jul 29 17:52:12.356872 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:12.356848    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6adbb4b3_7133_49d5_844b_182ed5602391.slice/crio-e204ff611238432cb97d076930d2fd5bfd643389d6b861b7a5eff4e5b42cde23 WatchSource:0}: Error finding container e204ff611238432cb97d076930d2fd5bfd643389d6b861b7a5eff4e5b42cde23: Status 404 returned error can't find the container with id e204ff611238432cb97d076930d2fd5bfd643389d6b861b7a5eff4e5b42cde23
Jul 29 17:52:12.364347 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.364326    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-52z29"
Jul 29 17:52:12.369131 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.369116    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:12.373466 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:12.373440    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76ae3d15_ef4e_4868_abdd_c86e08c8886c.slice/crio-9637bc7e4d0ab89020d02018a9ef4b050605b976fb4d1a897df74ee7d1614433 WatchSource:0}: Error finding container 9637bc7e4d0ab89020d02018a9ef4b050605b976fb4d1a897df74ee7d1614433: Status 404 returned error can't find the container with id 9637bc7e4d0ab89020d02018a9ef4b050605b976fb4d1a897df74ee7d1614433
Jul 29 17:52:12.378230 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:12.378211    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fa20546_4be3_4fe7_ba04_6428acad4a32.slice/crio-42c96c0d905570020455aa08d69709db47f43f1fcf00fd201338f9704d9608c5 WatchSource:0}: Error finding container 42c96c0d905570020455aa08d69709db47f43f1fcf00fd201338f9704d9608c5: Status 404 returned error can't find the container with id 42c96c0d905570020455aa08d69709db47f43f1fcf00fd201338f9704d9608c5
Jul 29 17:52:12.618445 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.618408    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:12.618617 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.618467    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:12.618617 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.618554    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:12.618617 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.618559    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:12.618617 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.618610    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret podName:9a962e36-54ac-40e3-8c51-c317c81f2133 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:13.618595491 +0000 UTC m=+3.126979060 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret") pod "global-pull-secret-syncer-5qf55" (UID: "9a962e36-54ac-40e3-8c51-c317c81f2133") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:12.618764 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.618625    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs podName:ad618126-a7f9-40d0-a92e-4b80729d8384 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:13.618618094 +0000 UTC m=+3.127001662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs") pod "network-metrics-daemon-vcnql" (UID: "ad618126-a7f9-40d0-a92e-4b80729d8384") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:12.711895 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.711816    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:12.719387 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.719333    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jnc\" (UniqueName: \"kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc\") pod \"network-check-target-pv5rx\" (UID: \"ced120c9-89f5-45bb-85b8-37d7e0b82f9e\") " pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:12.719532 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.719513    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:12.719586 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.719538    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:12.719586 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.719552    2556 projected.go:196] Error preparing data for projected volume kube-api-access-49jnc for pod openshift-network-diagnostics/network-check-target-pv5rx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:12.719669 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:12.719620    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc podName:ced120c9-89f5-45bb-85b8-37d7e0b82f9e nodeName:}" failed. No retries permitted until 2026-07-29 17:52:13.71960087 +0000 UTC m=+3.227984443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-49jnc" (UniqueName: "kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc") pod "network-check-target-pv5rx" (UID: "ced120c9-89f5-45bb-85b8-37d7e0b82f9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:12.971809 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.971639    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 17:47:12 +0000 UTC" deadline="2028-01-29 00:01:51.009561678 +0000 UTC"
Jul 29 17:52:12.971809 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:12.971702    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13158h9m38.037863849s"
Jul 29 17:52:13.096843 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.096713    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7hdr8" event={"ID":"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62","Type":"ContainerStarted","Data":"f1453035101ba2fe5ef2427148581051e2dcd5fa0e19cf2b20af35072564f6a3"}
Jul 29 17:52:13.114121 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.113839    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cttw7" event={"ID":"eb4927fd-bc0a-4316-b43f-fb9144018eab","Type":"ContainerStarted","Data":"bc691c142afe1d0d0629ad80d5f9cfce4333c7962ff0f61836ed2b7bc8fe7cf5"}
Jul 29 17:52:13.132286 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.132252    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jf97c" event={"ID":"3927a342-2b8b-47bd-bbb3-f78bbce3802a","Type":"ContainerStarted","Data":"3ff4dee80ccccaa8734e6b93a8e695c7822487d64a02b094d02af252a51bb967"}
Jul 29 17:52:13.149153 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.149114    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p69dr" event={"ID":"7f6cf20b-051e-4536-af42-e026c961f43b","Type":"ContainerStarted","Data":"347c9139979d5e0d295eb10814ae9fdec277e0e5686978eb39ab980a8876586b"}
Jul 29 17:52:13.160542 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.160271    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal" event={"ID":"e9a47ae04fab76d00c2743727b114e18","Type":"ContainerStarted","Data":"866c3b6b0639faf0b0b2ba3068a83b05cd167a8767c80e26b0ee5c19800c1720"}
Jul 29 17:52:13.179038 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.178830    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal" event={"ID":"2b348111e1b2cbede8ff3b7ecfdb4a2f","Type":"ContainerStarted","Data":"50e3aa52896f30080aa7ea76330e650f0cf283048a7d3c38700abcd1f26a2570"}
Jul 29 17:52:13.201283 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.201251    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-52z29" event={"ID":"76ae3d15-ef4e-4868-abdd-c86e08c8886c","Type":"ContainerStarted","Data":"9637bc7e4d0ab89020d02018a9ef4b050605b976fb4d1a897df74ee7d1614433"}
Jul 29 17:52:13.226501 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.225622    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkw8q" event={"ID":"6adbb4b3-7133-49d5-844b-182ed5602391","Type":"ContainerStarted","Data":"e204ff611238432cb97d076930d2fd5bfd643389d6b861b7a5eff4e5b42cde23"}
Jul 29 17:52:13.236589 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.236558    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l" event={"ID":"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117","Type":"ContainerStarted","Data":"3d8bc0fc045421100e3e8041c1d6141dedf0b773fc2c7b247f62030758bfa8b4"}
Jul 29 17:52:13.273406 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.270204    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2sqk8" event={"ID":"026c3a54-f3f1-4f0f-88d6-681bd039200d","Type":"ContainerStarted","Data":"9d5e5b23c0a272115f3949ffda940fc756bf4f2feec7889da1635278816a9130"}
Jul 29 17:52:13.286751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.286715    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx" event={"ID":"6fa20546-4be3-4fe7-ba04-6428acad4a32","Type":"ContainerStarted","Data":"42c96c0d905570020455aa08d69709db47f43f1fcf00fd201338f9704d9608c5"}
Jul 29 17:52:13.367806 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.367761    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:13.629707 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.629669    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:13.629895 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.629747    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:13.629976 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:13.629902    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:13.629976 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:13.629973    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs podName:ad618126-a7f9-40d0-a92e-4b80729d8384 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:15.629953523 +0000 UTC m=+5.138337095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs") pod "network-metrics-daemon-vcnql" (UID: "ad618126-a7f9-40d0-a92e-4b80729d8384") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:13.630418 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:13.630396    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:13.630546 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:13.630471    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret podName:9a962e36-54ac-40e3-8c51-c317c81f2133 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:15.630453879 +0000 UTC m=+5.138837451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret") pod "global-pull-secret-syncer-5qf55" (UID: "9a962e36-54ac-40e3-8c51-c317c81f2133") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:13.731254 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.731088    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jnc\" (UniqueName: \"kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc\") pod \"network-check-target-pv5rx\" (UID: \"ced120c9-89f5-45bb-85b8-37d7e0b82f9e\") " pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:13.731254 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:13.731257    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:13.731537 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:13.731280    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:13.731537 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:13.731293    2556 projected.go:196] Error preparing data for projected volume kube-api-access-49jnc for pod openshift-network-diagnostics/network-check-target-pv5rx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:13.731537 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:13.731367    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc podName:ced120c9-89f5-45bb-85b8-37d7e0b82f9e nodeName:}" failed. No retries permitted until 2026-07-29 17:52:15.731330232 +0000 UTC m=+5.239713805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-49jnc" (UniqueName: "kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc") pod "network-check-target-pv5rx" (UID: "ced120c9-89f5-45bb-85b8-37d7e0b82f9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:13.972918 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.972821    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 17:47:12 +0000 UTC" deadline="2028-04-22 05:19:46.565874358 +0000 UTC"
Jul 29 17:52:13.972918 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:13.972865    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15179h27m32.593014133s"
Jul 29 17:52:14.073221 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:14.073184    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:14.073409 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:14.073329    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:14.073788 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:14.073765    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:14.074028 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:14.073875    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:14.074028 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:14.073946    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:14.074149 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:14.074028    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:14.089624 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:14.089591    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:15.649846 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:15.648984    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:15.649846 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:15.649076    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:15.649846 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:15.649219    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:15.649846 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:15.649297    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret podName:9a962e36-54ac-40e3-8c51-c317c81f2133 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:19.649276348 +0000 UTC m=+9.157659920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret") pod "global-pull-secret-syncer-5qf55" (UID: "9a962e36-54ac-40e3-8c51-c317c81f2133") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:15.649846 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:15.649754    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:15.649846 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:15.649810    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs podName:ad618126-a7f9-40d0-a92e-4b80729d8384 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:19.649792767 +0000 UTC m=+9.158176340 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs") pod "network-metrics-daemon-vcnql" (UID: "ad618126-a7f9-40d0-a92e-4b80729d8384") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:15.750857 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:15.750112    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jnc\" (UniqueName: \"kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc\") pod \"network-check-target-pv5rx\" (UID: \"ced120c9-89f5-45bb-85b8-37d7e0b82f9e\") " pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:15.750857 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:15.750342    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:15.750857 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:15.750369    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:15.750857 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:15.750384    2556 projected.go:196] Error preparing data for projected volume kube-api-access-49jnc for pod openshift-network-diagnostics/network-check-target-pv5rx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:15.750857 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:15.750472    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc podName:ced120c9-89f5-45bb-85b8-37d7e0b82f9e nodeName:}" failed. No retries permitted until 2026-07-29 17:52:19.750449803 +0000 UTC m=+9.258833409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-49jnc" (UniqueName: "kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc") pod "network-check-target-pv5rx" (UID: "ced120c9-89f5-45bb-85b8-37d7e0b82f9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:16.073220 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:16.073185    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:16.073459 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:16.073320    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:16.073791 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:16.073744    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:16.073882 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:16.073857    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:16.073965 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:16.073935    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:16.074145 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:16.074022    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:18.072938 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:18.072903    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:18.073464 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:18.073132    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:18.073464 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:18.073207    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:18.073464 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:18.073314    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:18.073464 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:18.073314    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:18.073717 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:18.073530    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:19.684609 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:19.684534    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:19.685224 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:19.684617    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:19.685224 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:19.684748    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:19.685224 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:19.684811    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret podName:9a962e36-54ac-40e3-8c51-c317c81f2133 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:27.684792653 +0000 UTC m=+17.193176228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret") pod "global-pull-secret-syncer-5qf55" (UID: "9a962e36-54ac-40e3-8c51-c317c81f2133") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:19.685224 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:19.685212    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:19.685543 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:19.685263    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs podName:ad618126-a7f9-40d0-a92e-4b80729d8384 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:27.685246714 +0000 UTC m=+17.193630282 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs") pod "network-metrics-daemon-vcnql" (UID: "ad618126-a7f9-40d0-a92e-4b80729d8384") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:19.786163 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:19.786021    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jnc\" (UniqueName: \"kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc\") pod \"network-check-target-pv5rx\" (UID: \"ced120c9-89f5-45bb-85b8-37d7e0b82f9e\") " pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:19.786328 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:19.786282    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:19.786328 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:19.786304    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:19.786328 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:19.786318    2556 projected.go:196] Error preparing data for projected volume kube-api-access-49jnc for pod openshift-network-diagnostics/network-check-target-pv5rx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:19.786521 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:19.786383    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc podName:ced120c9-89f5-45bb-85b8-37d7e0b82f9e nodeName:}" failed. No retries permitted until 2026-07-29 17:52:27.786361917 +0000 UTC m=+17.294745507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-49jnc" (UniqueName: "kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc") pod "network-check-target-pv5rx" (UID: "ced120c9-89f5-45bb-85b8-37d7e0b82f9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:20.073623 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:20.073187    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:20.073623 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:20.073336    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:20.073623 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:20.073470    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:20.073623 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:20.073335    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:20.073623 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:20.073548    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:20.074048 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:20.073678    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:22.073805 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:22.073762    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:22.074294 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:22.073903    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:22.074470 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:22.074422    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:22.074608 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:22.074555    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:22.074681 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:22.074609    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:22.074740 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:22.074682    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:24.072836 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:24.072799    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:24.073270 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:24.072799    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:24.073270 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:24.072940    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:24.073270 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:24.072799    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:24.073270 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:24.072983    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:24.073270 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:24.073054    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:26.073126 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:26.073087    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:26.073624 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:26.073129    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:26.073624 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:26.073130    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:26.073624 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:26.073218    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:26.073624 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:26.073336    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:26.073624 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:26.073456    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:27.749377 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:27.749336    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:27.749847 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:27.749428    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:27.749847 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:27.749501    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:27.749847 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:27.749535    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:27.749847 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:27.749577    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs podName:ad618126-a7f9-40d0-a92e-4b80729d8384 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:43.749555363 +0000 UTC m=+33.257938945 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs") pod "network-metrics-daemon-vcnql" (UID: "ad618126-a7f9-40d0-a92e-4b80729d8384") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:27.749847 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:27.749595    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret podName:9a962e36-54ac-40e3-8c51-c317c81f2133 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:43.749586405 +0000 UTC m=+33.257969984 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret") pod "global-pull-secret-syncer-5qf55" (UID: "9a962e36-54ac-40e3-8c51-c317c81f2133") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:27.850334 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:27.850300    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jnc\" (UniqueName: \"kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc\") pod \"network-check-target-pv5rx\" (UID: \"ced120c9-89f5-45bb-85b8-37d7e0b82f9e\") " pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:27.850519 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:27.850469    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:27.850519 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:27.850491    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:27.850519 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:27.850514    2556 projected.go:196] Error preparing data for projected volume kube-api-access-49jnc for pod openshift-network-diagnostics/network-check-target-pv5rx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:27.850668 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:27.850577    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc podName:ced120c9-89f5-45bb-85b8-37d7e0b82f9e nodeName:}" failed. No retries permitted until 2026-07-29 17:52:43.850556648 +0000 UTC m=+33.358940237 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-49jnc" (UniqueName: "kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc") pod "network-check-target-pv5rx" (UID: "ced120c9-89f5-45bb-85b8-37d7e0b82f9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:28.073693 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:28.073596    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:28.073693 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:28.073647    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:28.073693 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:28.073647    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:28.073949 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:28.073726    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:28.073949 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:28.073871    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:28.074036 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:28.073979    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:30.073686 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:30.073646    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:30.074110 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:30.073663    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:30.074110 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:30.073772    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:30.074110 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:30.073783    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:30.074110 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:30.073853    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:30.074110 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:30.073930    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:32.072857 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:32.072781    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:32.073333 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:32.072781    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:32.073333 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:32.072934    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:32.073333 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:32.072781    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:32.073333 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:32.073020    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:32.073333 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:32.073087    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:33.355569 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:33.355165    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx" event={"ID":"6fa20546-4be3-4fe7-ba04-6428acad4a32","Type":"ContainerStarted","Data":"fee196335bce57391aee56fa8a659aef08f62054b815604ec83aa2d08c300d33"}
Jul 29 17:52:33.360537 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:33.360408    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7hdr8" event={"ID":"7c9c6ced-e28d-42f1-88a1-9a06ec53ba62","Type":"ContainerStarted","Data":"8e2bbd63262c1c6d2b25bd6fa21e6998f5f27fcc16faeefd8cbed2dafdb7dd05"}
Jul 29 17:52:33.368942 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:33.368377    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p69dr" event={"ID":"7f6cf20b-051e-4536-af42-e026c961f43b","Type":"ContainerStarted","Data":"90948c235369d89bd88ca6709175923f8d5188ec721b6221ead7804b2942694c"}
Jul 29 17:52:33.371218 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:33.371073    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal" event={"ID":"e9a47ae04fab76d00c2743727b114e18","Type":"ContainerStarted","Data":"24fd52eee65cd101d490737242640ef153f65cda5cd753f1cd22bb7b049466bd"}
Jul 29 17:52:33.380011 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:33.379959    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7hdr8" podStartSLOduration=0.67394858 podStartE2EDuration="21.379937571s" totalImagesPullingTime="20.705988991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:33.378780924 +0000 UTC m=+22.887164510" watchObservedRunningTime="2026-07-29 17:52:33.379937571 +0000 UTC m=+22.888321176"
Jul 29 17:52:33.412592 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:33.410852    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-253.ec2.internal" podStartSLOduration=21.410832555 podStartE2EDuration="21.410832555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:33.393491118 +0000 UTC m=+22.901874708" watchObservedRunningTime="2026-07-29 17:52:33.410832555 +0000 UTC m=+22.919216143"
Jul 29 17:52:33.412592 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:33.411062    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p69dr" podStartSLOduration=0.657869787 podStartE2EDuration="21.411055097s" totalImagesPullingTime="20.75318531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:33.410051084 +0000 UTC m=+22.918434674" watchObservedRunningTime="2026-07-29 17:52:33.411055097 +0000 UTC m=+22.919438687"
Jul 29 17:52:34.073118 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.073034    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:34.073118 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.073078    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:34.073118 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.073078    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:34.073412 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:34.073154    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:34.073412 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:34.073274    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:34.073412 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:34.073376    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:34.374724 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.374647    2556 generic.go:356] "Generic (PLEG): container finished" podID="6adbb4b3-7133-49d5-844b-182ed5602391" containerID="053ceb57828fb2aa805d407ff650441799e35f4425a66b62a524cccbb9d0f9a2" exitCode=0
Jul 29 17:52:34.375558 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.374722    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkw8q" event={"ID":"6adbb4b3-7133-49d5-844b-182ed5602391","Type":"ContainerDied","Data":"053ceb57828fb2aa805d407ff650441799e35f4425a66b62a524cccbb9d0f9a2"}
Jul 29 17:52:34.376550 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.376523    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l" event={"ID":"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117","Type":"ContainerStarted","Data":"3ffd4f9462a81aff3135135446639ead14a92f09f09cdf27cc7c084ce66ff4fc"}
Jul 29 17:52:34.378161 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.378133    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2sqk8" event={"ID":"026c3a54-f3f1-4f0f-88d6-681bd039200d","Type":"ContainerStarted","Data":"99915e3cba3d97ddeb5b73f2f7967e3733d540d81a3d65b2a4b225ea19e70f9d"}
Jul 29 17:52:34.381791 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.381767    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx" event={"ID":"6fa20546-4be3-4fe7-ba04-6428acad4a32","Type":"ContainerStarted","Data":"5db5ad68670ac90bd7c5fd9114a18e26b2b07f0ac3e4d0398a5355cd0d6bedc8"}
Jul 29 17:52:34.381899 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.381841    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx" event={"ID":"6fa20546-4be3-4fe7-ba04-6428acad4a32","Type":"ContainerStarted","Data":"2de87dbcc94dbfd560dd6ec2372c40a7fce8e445db42aefced17927df9af8075"}
Jul 29 17:52:34.381899 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.381891    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx" event={"ID":"6fa20546-4be3-4fe7-ba04-6428acad4a32","Type":"ContainerStarted","Data":"74cc4cdf2012f9dc96280372ffc24f166c6478d7ee942aa2d0da0ea40e4af30f"}
Jul 29 17:52:34.382043 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.381938    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx" event={"ID":"6fa20546-4be3-4fe7-ba04-6428acad4a32","Type":"ContainerStarted","Data":"8db3390e962d46b9045c416ffff2a2dbe6edaed5d4a17d26661120d82575ac56"}
Jul 29 17:52:34.382043 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.381985    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx" event={"ID":"6fa20546-4be3-4fe7-ba04-6428acad4a32","Type":"ContainerStarted","Data":"f9d110c8a9429d883c25344571b987d8174f22df852daef9bee569a0badf3407"}
Jul 29 17:52:34.383370 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.383330    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cttw7" event={"ID":"eb4927fd-bc0a-4316-b43f-fb9144018eab","Type":"ContainerStarted","Data":"140415f2afd2aa7b524b16d4414d5cb9dccf6ff777c3f21539f9e198f95f3f95"}
Jul 29 17:52:34.385153 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.385130    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jf97c" event={"ID":"3927a342-2b8b-47bd-bbb3-f78bbce3802a","Type":"ContainerStarted","Data":"d764833966d962178a7a60565e6043b68b30d40bfa903ab32cf434bf2beb63d2"}
Jul 29 17:52:34.386716 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.386689    2556 generic.go:356] "Generic (PLEG): container finished" podID="2b348111e1b2cbede8ff3b7ecfdb4a2f" containerID="1fdcf168d8c834ba1b2792ed8f176d1c2d0fe8cfa1a42d263e6ab502505ce1db" exitCode=0
Jul 29 17:52:34.386812 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.386762    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal" event={"ID":"2b348111e1b2cbede8ff3b7ecfdb4a2f","Type":"ContainerDied","Data":"1fdcf168d8c834ba1b2792ed8f176d1c2d0fe8cfa1a42d263e6ab502505ce1db"}
Jul 29 17:52:34.411032 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.410990    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jf97c" podStartSLOduration=1.6787210780000001 podStartE2EDuration="22.410975082s" totalImagesPullingTime="20.732254004s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:34.410675609 +0000 UTC m=+23.919059196" watchObservedRunningTime="2026-07-29 17:52:34.410975082 +0000 UTC m=+23.919358669"
Jul 29 17:52:34.439740 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.439703    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2sqk8" podStartSLOduration=1.698769277 podStartE2EDuration="22.439688161s" totalImagesPullingTime="20.740918884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:34.439472096 +0000 UTC m=+23.947855683" watchObservedRunningTime="2026-07-29 17:52:34.439688161 +0000 UTC m=+23.948071748"
Jul 29 17:52:34.453329 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.453294    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cttw7" podStartSLOduration=1.994183571 podStartE2EDuration="22.453281692s" totalImagesPullingTime="20.459098121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:34.453241249 +0000 UTC m=+23.961624836" watchObservedRunningTime="2026-07-29 17:52:34.453281692 +0000 UTC m=+23.961665280"
Jul 29 17:52:34.753347 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:34.753182    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 17:52:35.111837 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:35.111674    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T17:52:34.753342812Z","UUID":"b6cbfe56-d22a-4029-b898-96085edeb12e","Handler":null,"Name":"","Endpoint":""}
Jul 29 17:52:35.113405 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:35.113378    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 17:52:35.113541 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:35.113426    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 17:52:35.390745 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:35.390700    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-52z29" event={"ID":"76ae3d15-ef4e-4868-abdd-c86e08c8886c","Type":"ContainerStarted","Data":"cd9933ee1ccf1728ee50844b814393337df1cf8f9b87a2528e8ad11ee882e1d3"}
Jul 29 17:52:35.392813 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:35.392787    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l" event={"ID":"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117","Type":"ContainerStarted","Data":"28c3340446fcafc1ec7c1e6e55069eea072bc5b47762ce911bf882bd757bded0"}
Jul 29 17:52:35.395222 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:35.395196    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal" event={"ID":"2b348111e1b2cbede8ff3b7ecfdb4a2f","Type":"ContainerStarted","Data":"5659fa3027954c3b5b86fd10e0a683ae97c5c74128a00efc9448418139d41061"}
Jul 29 17:52:35.407968 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:35.407918    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-52z29" podStartSLOduration=2.729766099 podStartE2EDuration="23.40790426s" totalImagesPullingTime="20.678138161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:35.407445275 +0000 UTC m=+24.915828864" watchObservedRunningTime="2026-07-29 17:52:35.40790426 +0000 UTC m=+24.916287847"
Jul 29 17:52:35.424571 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:35.424524    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-253.ec2.internal" podStartSLOduration=23.424505293 podStartE2EDuration="23.424505293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:35.424273007 +0000 UTC m=+24.932656597" watchObservedRunningTime="2026-07-29 17:52:35.424505293 +0000 UTC m=+24.932888878"
Jul 29 17:52:35.561013 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:35.560974    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:35.688886 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:35.688818    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:36.073677 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:36.073649    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:36.073964 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:36.073690    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:36.073964 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:36.073649    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:36.073964 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:36.073779    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:36.073964 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:36.073892    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:36.074206 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:36.073966    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:36.401000 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:36.400964    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx" event={"ID":"6fa20546-4be3-4fe7-ba04-6428acad4a32","Type":"ContainerStarted","Data":"eb8ca50248c7837708e6b9c0877055fc751caa3535a2cb2732269a6616cc1d1c"}
Jul 29 17:52:36.403165 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:36.403131    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l" event={"ID":"9e08cf6d-6e08-4ebb-8c6c-460dca8e8117","Type":"ContainerStarted","Data":"255b2f6aa52ba3d3fdcb059d56fed9038bac3e79c76499f07e9001385aa15f8b"}
Jul 29 17:52:36.423599 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:36.423554    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tl8l" podStartSLOduration=1.5744994989999999 podStartE2EDuration="24.423538005s" totalImagesPullingTime="22.849038506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:36.423091873 +0000 UTC m=+25.931475464" watchObservedRunningTime="2026-07-29 17:52:36.423538005 +0000 UTC m=+25.931921593"
Jul 29 17:52:36.791986 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:36.791915    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:36.879942 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:36.879908    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:38.073493 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:38.072749    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:38.073493 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:38.072783    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:38.073493 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:38.072852    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:38.073493 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:38.072749    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:38.073493 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:38.072943    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:38.073493 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:38.073026    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:38.229078 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:38.229048    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:38.229704 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:38.229685    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:38.407879 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:38.407846    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:38.408340 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:38.408323    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2sqk8"
Jul 29 17:52:38.468301 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:38.468265    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:39.006049 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:39.005873    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:39.066835 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:39.066774    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:39.130962 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:39.130934    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:39.413417 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:39.413377    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx" event={"ID":"6fa20546-4be3-4fe7-ba04-6428acad4a32","Type":"ContainerStarted","Data":"7d0d9f9ae2641c6faffb58a1fa5ad33b73a1a7bb86cc2b879810225477de68ce"}
Jul 29 17:52:39.413599 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:39.413590    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:39.415261 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:39.415232    2556 generic.go:356] "Generic (PLEG): container finished" podID="6adbb4b3-7133-49d5-844b-182ed5602391" containerID="741a1a007fc64854767e39ffcf008826ce519a40fa12c62e498d960797f7791c" exitCode=0
Jul 29 17:52:39.415416 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:39.415318    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkw8q" event={"ID":"6adbb4b3-7133-49d5-844b-182ed5602391","Type":"ContainerDied","Data":"741a1a007fc64854767e39ffcf008826ce519a40fa12c62e498d960797f7791c"}
Jul 29 17:52:39.428872 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:39.428849    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:39.446074 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:39.446038    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx" podStartSLOduration=6.644939065 podStartE2EDuration="27.446024999s" totalImagesPullingTime="20.801085934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:39.445844461 +0000 UTC m=+28.954228049" watchObservedRunningTime="2026-07-29 17:52:39.446024999 +0000 UTC m=+28.954408586"
Jul 29 17:52:39.696296 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:39.696214    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:39.755131 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:39.755102    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:40.073465 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.073393    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:40.073465 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.073419    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:40.073465 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.073420    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:40.073835 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:40.073547    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:40.073835 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:40.073679    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:40.073835 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:40.073764    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:40.418210 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.417948    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:40.418757 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.418394    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:40.433266 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.433244    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:52:40.557921 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.557887    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pv5rx"]
Jul 29 17:52:40.558062 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.557992    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:40.558129 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:40.558097    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:40.558574 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.558550    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5qf55"]
Jul 29 17:52:40.558684 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.558628    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:40.558754 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:40.558696    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:40.560686 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.560664    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vcnql"]
Jul 29 17:52:40.560772 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:40.560735    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:40.560840 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:40.560813    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:41.422240 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:41.422210    2556 generic.go:356] "Generic (PLEG): container finished" podID="6adbb4b3-7133-49d5-844b-182ed5602391" containerID="d43937b1c53cfcec458eb1e90846f68ee17b1a0806a6f4240fe44bcf2cc982d8" exitCode=0
Jul 29 17:52:41.422639 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:41.422294    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkw8q" event={"ID":"6adbb4b3-7133-49d5-844b-182ed5602391","Type":"ContainerDied","Data":"d43937b1c53cfcec458eb1e90846f68ee17b1a0806a6f4240fe44bcf2cc982d8"}
Jul 29 17:52:41.579230 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:41.579202    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:41.681165 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:41.681079    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:41.834204 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:41.834152    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:41.936822 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:41.936753    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:42.073557 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:42.073524    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:42.073700 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:42.073559    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:42.073700 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:42.073563    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:42.073700 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:42.073627    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:42.073876 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:42.073746    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:42.073876 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:42.073841    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:43.429033 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:43.429002    2556 generic.go:356] "Generic (PLEG): container finished" podID="6adbb4b3-7133-49d5-844b-182ed5602391" containerID="0c2bc84dbc4eca468740d8e6f19b60f45f8cbec59779675a539fa1dc7c6fed07" exitCode=0
Jul 29 17:52:43.429549 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:43.429043    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkw8q" event={"ID":"6adbb4b3-7133-49d5-844b-182ed5602391","Type":"ContainerDied","Data":"0c2bc84dbc4eca468740d8e6f19b60f45f8cbec59779675a539fa1dc7c6fed07"}
Jul 29 17:52:43.766899 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:43.766832    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:43.766899 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:43.766879    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:43.767063 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:43.766974    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:43.767063 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:43.767028    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret podName:9a962e36-54ac-40e3-8c51-c317c81f2133 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:15.767014129 +0000 UTC m=+65.275397699 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret") pod "global-pull-secret-syncer-5qf55" (UID: "9a962e36-54ac-40e3-8c51-c317c81f2133") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:43.767136 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:43.766975    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:43.767136 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:43.767113    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs podName:ad618126-a7f9-40d0-a92e-4b80729d8384 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:15.767101068 +0000 UTC m=+65.275484648 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs") pod "network-metrics-daemon-vcnql" (UID: "ad618126-a7f9-40d0-a92e-4b80729d8384") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:43.867474 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:43.867449    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jnc\" (UniqueName: \"kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc\") pod \"network-check-target-pv5rx\" (UID: \"ced120c9-89f5-45bb-85b8-37d7e0b82f9e\") " pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:43.867657 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:43.867639    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:43.867695 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:43.867664    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:43.867695 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:43.867674    2556 projected.go:196] Error preparing data for projected volume kube-api-access-49jnc for pod openshift-network-diagnostics/network-check-target-pv5rx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:43.867762 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:43.867726    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc podName:ced120c9-89f5-45bb-85b8-37d7e0b82f9e nodeName:}" failed. No retries permitted until 2026-07-29 17:53:15.867713352 +0000 UTC m=+65.376096917 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-49jnc" (UniqueName: "kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc") pod "network-check-target-pv5rx" (UID: "ced120c9-89f5-45bb-85b8-37d7e0b82f9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:44.073798 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:44.073727    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:44.073798 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:44.073768    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:44.073798 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:44.073768    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:44.074051 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:44.073835    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qf55" podUID="9a962e36-54ac-40e3-8c51-c317c81f2133"
Jul 29 17:52:44.074051 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:44.073937    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vcnql" podUID="ad618126-a7f9-40d0-a92e-4b80729d8384"
Jul 29 17:52:44.074051 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:44.074002    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pv5rx" podUID="ced120c9-89f5-45bb-85b8-37d7e0b82f9e"
Jul 29 17:52:45.842661 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.842633    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-253.ec2.internal" event="NodeReady"
Jul 29 17:52:45.843085 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.842766    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 17:52:45.878189 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.878157    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67f78b6ccd-vj58l"]
Jul 29 17:52:45.881798 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.881776    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:45.884551 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.884523    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 17:52:45.884798 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.884775    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 17:52:45.884915 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.884778    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 17:52:45.884915 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.884783    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w8nx9\""
Jul 29 17:52:45.891293 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.891141    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8v82s"]
Jul 29 17:52:45.892312 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.892290    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 17:52:45.896685 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.895515    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jhjsc"]
Jul 29 17:52:45.899967 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.898525    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:45.899967 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.899784    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67f78b6ccd-vj58l"]
Jul 29 17:52:45.899967 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.899889    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jhjsc"
Jul 29 17:52:45.901765 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.901740    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tr8hk\""
Jul 29 17:52:45.902647 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.902252    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 17:52:45.902647 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.902482    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 17:52:45.902647 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.902593    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jhjsc"]
Jul 29 17:52:45.902845 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.902731    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nmb94\""
Jul 29 17:52:45.902921 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.902903    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 17:52:45.902972 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.902912    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 17:52:45.903351 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.903183    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 17:52:45.906001 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.905984    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8v82s"]
Jul 29 17:52:45.983152 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983104    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-ca-trust-extracted\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:45.983152 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983148    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-bound-sa-token\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:45.983403 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983174    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-installation-pull-secrets\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:45.983403 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983220    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv5cv\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-kube-api-access-xv5cv\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:45.983403 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983267    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-image-registry-private-configuration\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:45.983403 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983287    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfnk2\" (UniqueName: \"kubernetes.io/projected/37daa72f-68a5-462f-87f4-cbe6981c77a0-kube-api-access-qfnk2\") pod \"ingress-canary-jhjsc\" (UID: \"37daa72f-68a5-462f-87f4-cbe6981c77a0\") " pod="openshift-ingress-canary/ingress-canary-jhjsc"
Jul 29 17:52:45.983403 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983305    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-tmp-dir\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:45.983642 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983408    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-certificates\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:45.983642 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983433    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37daa72f-68a5-462f-87f4-cbe6981c77a0-cert\") pod \"ingress-canary-jhjsc\" (UID: \"37daa72f-68a5-462f-87f4-cbe6981c77a0\") " pod="openshift-ingress-canary/ingress-canary-jhjsc"
Jul 29 17:52:45.983642 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983456    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-config-volume\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:45.983642 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983475    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-trusted-ca\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:45.983642 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983490    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btcg7\" (UniqueName: \"kubernetes.io/projected/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-kube-api-access-btcg7\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:45.983642 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983541    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:45.983642 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:45.983612    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:46.038424 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.038392    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:46.073497 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.073462    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:52:46.073497 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.073482    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:52:46.073706 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.073482    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:52:46.076643 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.076588    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 17:52:46.076643 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.076601    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 17:52:46.076898 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.076651    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w7s6p\""
Jul 29 17:52:46.076898 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.076721    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-llfsb\""
Jul 29 17:52:46.076898 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.076857    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 17:52:46.077028 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.076942    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 17:52:46.084345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084320    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-installation-pull-secrets\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.084489 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084381    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv5cv\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-kube-api-access-xv5cv\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.084489 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084423    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-image-registry-private-configuration\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.084489 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084450    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfnk2\" (UniqueName: \"kubernetes.io/projected/37daa72f-68a5-462f-87f4-cbe6981c77a0-kube-api-access-qfnk2\") pod \"ingress-canary-jhjsc\" (UID: \"37daa72f-68a5-462f-87f4-cbe6981c77a0\") " pod="openshift-ingress-canary/ingress-canary-jhjsc"
Jul 29 17:52:46.084489 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084470    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-tmp-dir\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:46.084709 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084530    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-certificates\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.084709 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084555    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37daa72f-68a5-462f-87f4-cbe6981c77a0-cert\") pod \"ingress-canary-jhjsc\" (UID: \"37daa72f-68a5-462f-87f4-cbe6981c77a0\") " pod="openshift-ingress-canary/ingress-canary-jhjsc"
Jul 29 17:52:46.084709 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084579    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-config-volume\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:46.084709 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084605    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-trusted-ca\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.084709 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084627    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btcg7\" (UniqueName: \"kubernetes.io/projected/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-kube-api-access-btcg7\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:46.084709 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084653    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.084709 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084686    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:46.085034 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084747    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-ca-trust-extracted\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.085034 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084777    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-bound-sa-token\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.085034 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.084871    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-tmp-dir\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:46.085034 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:46.085004    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 17:52:46.085236 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:46.085070    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls podName:dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e nodeName:}" failed. No retries permitted until 2026-07-29 17:52:46.585050458 +0000 UTC m=+36.093434041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls") pod "dns-default-8v82s" (UID: "dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e") : secret "dns-default-metrics-tls" not found
Jul 29 17:52:46.085339 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:46.085316    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 17:52:46.085436 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:46.085341    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f78b6ccd-vj58l: secret "image-registry-tls" not found
Jul 29 17:52:46.085436 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.085396    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-config-volume\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:46.085436 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:46.085402    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls podName:2a4db46e-c7d3-4b21-935e-b2e79d07c5cf nodeName:}" failed. No retries permitted until 2026-07-29 17:52:46.585386165 +0000 UTC m=+36.093769744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls") pod "image-registry-67f78b6ccd-vj58l" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf") : secret "image-registry-tls" not found
Jul 29 17:52:46.085600 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.085507    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-certificates\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.085655 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.085608    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-ca-trust-extracted\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.085895 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.085873    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-trusted-ca\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.089559 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.089409    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37daa72f-68a5-462f-87f4-cbe6981c77a0-cert\") pod \"ingress-canary-jhjsc\" (UID: \"37daa72f-68a5-462f-87f4-cbe6981c77a0\") " pod="openshift-ingress-canary/ingress-canary-jhjsc"
Jul 29 17:52:46.089666 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.089439    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-installation-pull-secrets\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.089666 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.089439    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-image-registry-private-configuration\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.093772 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.093742    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btcg7\" (UniqueName: \"kubernetes.io/projected/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-kube-api-access-btcg7\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:46.094804 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.094710    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfnk2\" (UniqueName: \"kubernetes.io/projected/37daa72f-68a5-462f-87f4-cbe6981c77a0-kube-api-access-qfnk2\") pod \"ingress-canary-jhjsc\" (UID: \"37daa72f-68a5-462f-87f4-cbe6981c77a0\") " pod="openshift-ingress-canary/ingress-canary-jhjsc"
Jul 29 17:52:46.095120 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.095100    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv5cv\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-kube-api-access-xv5cv\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.095348 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.095329    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-bound-sa-token\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.140411 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.140332    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:46.218720 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.218683    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jhjsc"
Jul 29 17:52:46.246611 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.246579    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:46.588853 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.588775    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:46.589030 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:46.588899    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:46.589030 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:46.588946    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 17:52:46.589030 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:46.589007    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 17:52:46.589030 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:46.589019    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f78b6ccd-vj58l: secret "image-registry-tls" not found
Jul 29 17:52:46.589030 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:46.589022    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls podName:dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e nodeName:}" failed. No retries permitted until 2026-07-29 17:52:47.589003136 +0000 UTC m=+37.097386716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls") pod "dns-default-8v82s" (UID: "dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e") : secret "dns-default-metrics-tls" not found
Jul 29 17:52:46.589271 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:46.589081    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls podName:2a4db46e-c7d3-4b21-935e-b2e79d07c5cf nodeName:}" failed. No retries permitted until 2026-07-29 17:52:47.589062667 +0000 UTC m=+37.097446258 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls") pod "image-registry-67f78b6ccd-vj58l" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf") : secret "image-registry-tls" not found
Jul 29 17:52:47.597854 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:47.597793    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:47.598614 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:47.597872    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:47.598614 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:47.597962    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 17:52:47.598614 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:47.597983    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f78b6ccd-vj58l: secret "image-registry-tls" not found
Jul 29 17:52:47.598614 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:47.598053    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls podName:2a4db46e-c7d3-4b21-935e-b2e79d07c5cf nodeName:}" failed. No retries permitted until 2026-07-29 17:52:49.598023386 +0000 UTC m=+39.106406974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls") pod "image-registry-67f78b6ccd-vj58l" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf") : secret "image-registry-tls" not found
Jul 29 17:52:47.598614 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:47.598124    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 17:52:47.598614 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:47.598213    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls podName:dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e nodeName:}" failed. No retries permitted until 2026-07-29 17:52:49.598196261 +0000 UTC m=+39.106579855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls") pod "dns-default-8v82s" (UID: "dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e") : secret "dns-default-metrics-tls" not found
Jul 29 17:52:47.661644 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:47.661616    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jhjsc"]
Jul 29 17:52:47.664914 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:47.664881    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37daa72f_68a5_462f_87f4_cbe6981c77a0.slice/crio-11d520105c13409e7ca6a498568e4c0e64749a37a09349406ff98a56e0785550 WatchSource:0}: Error finding container 11d520105c13409e7ca6a498568e4c0e64749a37a09349406ff98a56e0785550: Status 404 returned error can't find the container with id 11d520105c13409e7ca6a498568e4c0e64749a37a09349406ff98a56e0785550
Jul 29 17:52:48.442318 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:48.442279    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jhjsc" event={"ID":"37daa72f-68a5-462f-87f4-cbe6981c77a0","Type":"ContainerStarted","Data":"11d520105c13409e7ca6a498568e4c0e64749a37a09349406ff98a56e0785550"}
Jul 29 17:52:48.445402 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:48.445356    2556 generic.go:356] "Generic (PLEG): container finished" podID="6adbb4b3-7133-49d5-844b-182ed5602391" containerID="e1bf1dc65b94389c130848dd8938136bd4bb43c00a92d8c784814fbc2d942901" exitCode=0
Jul 29 17:52:48.445544 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:48.445394    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkw8q" event={"ID":"6adbb4b3-7133-49d5-844b-182ed5602391","Type":"ContainerDied","Data":"e1bf1dc65b94389c130848dd8938136bd4bb43c00a92d8c784814fbc2d942901"}
Jul 29 17:52:49.037971 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:49.037934    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:49.139658 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:49.139625    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:49.452470 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:49.452196    2556 generic.go:356] "Generic (PLEG): container finished" podID="6adbb4b3-7133-49d5-844b-182ed5602391" containerID="52165b3f0511484efd3827f0a0642150430d996053fe87f3b8c0bd03ee501ff8" exitCode=0
Jul 29 17:52:49.452470 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:49.452251    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkw8q" event={"ID":"6adbb4b3-7133-49d5-844b-182ed5602391","Type":"ContainerDied","Data":"52165b3f0511484efd3827f0a0642150430d996053fe87f3b8c0bd03ee501ff8"}
Jul 29 17:52:49.614209 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:49.614173    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:49.614355 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:49.614219    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:49.614355 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:49.614331    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 17:52:49.614355 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:49.614349    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f78b6ccd-vj58l: secret "image-registry-tls" not found
Jul 29 17:52:49.614547 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:49.614420    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls podName:2a4db46e-c7d3-4b21-935e-b2e79d07c5cf nodeName:}" failed. No retries permitted until 2026-07-29 17:52:53.614401013 +0000 UTC m=+43.122784579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls") pod "image-registry-67f78b6ccd-vj58l" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf") : secret "image-registry-tls" not found
Jul 29 17:52:49.614547 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:49.614469    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 17:52:49.614547 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:49.614520    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls podName:dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e nodeName:}" failed. No retries permitted until 2026-07-29 17:52:53.614503806 +0000 UTC m=+43.122887372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls") pod "dns-default-8v82s" (UID: "dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e") : secret "dns-default-metrics-tls" not found
Jul 29 17:52:50.229949 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.229916    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:50.235143 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.235093    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hzmb7"]
Jul 29 17:52:50.259505 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.259473    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hzmb7"]
Jul 29 17:52:50.259680 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.259607    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.262451 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.262252    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 17:52:50.262451 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.262285    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 17:52:50.262451 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.262293    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fvz84\""
Jul 29 17:52:50.262451 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.262401    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 17:52:50.262730 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.262574    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 17:52:50.309543 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.309508    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:50.377846 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.377810    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz"]
Jul 29 17:52:50.386057 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.386028    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz"
Jul 29 17:52:50.389081 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.389056    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-cbwkv\""
Jul 29 17:52:50.389604 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.389566    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:50.389843 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.389819    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 29 17:52:50.390017 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.389996    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz"]
Jul 29 17:52:50.408099 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.408074    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:50.419951 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.419923    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/26658a95-087c-47ea-9a71-bb1cc1f85801-data-volume\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.420126 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.419980    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.420126 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.420020    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.420126 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.420068    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dlvm\" (UniqueName: \"kubernetes.io/projected/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-api-access-6dlvm\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.420283 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.420170    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/26658a95-087c-47ea-9a71-bb1cc1f85801-crio-socket\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.457860 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.457832    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkw8q" event={"ID":"6adbb4b3-7133-49d5-844b-182ed5602391","Type":"ContainerStarted","Data":"492cc52493718e7a02ca60c3d96c4875a3246236c242f0ef57c5e0439cda5aae"}
Jul 29 17:52:50.482398 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.482324    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wkw8q" podStartSLOduration=7.313223732 podStartE2EDuration="38.482302575s" totalImagesPullingTime="31.169078843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:50.480272669 +0000 UTC m=+39.988656281" watchObservedRunningTime="2026-07-29 17:52:50.482302575 +0000 UTC m=+39.990686163"
Jul 29 17:52:50.487620 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.487594    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:50.520679 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.520652    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/26658a95-087c-47ea-9a71-bb1cc1f85801-crio-socket\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.520679 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.520682    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/26658a95-087c-47ea-9a71-bb1cc1f85801-data-volume\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.520892 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.520716    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.520892 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.520735    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.520892 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.520755    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpjc2\" (UniqueName: \"kubernetes.io/projected/a98b8bc6-2242-4c67-abca-75e6b55c5d1f-kube-api-access-tpjc2\") pod \"migrator-6865c9967-8zkjz\" (UID: \"a98b8bc6-2242-4c67-abca-75e6b55c5d1f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz"
Jul 29 17:52:50.520892 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.520779    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlvm\" (UniqueName: \"kubernetes.io/projected/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-api-access-6dlvm\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.520892 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:50.520791    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 17:52:50.520892 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:50.520868    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-rbac-proxy-cm podName:26658a95-087c-47ea-9a71-bb1cc1f85801 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:51.020847467 +0000 UTC m=+40.529231053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hzmb7" (UID: "26658a95-087c-47ea-9a71-bb1cc1f85801") : configmap "kube-rbac-proxy" not found
Jul 29 17:52:50.521157 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:50.520897    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:50.521157 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:50.520961    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls podName:26658a95-087c-47ea-9a71-bb1cc1f85801 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:51.020944253 +0000 UTC m=+40.529327834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hzmb7" (UID: "26658a95-087c-47ea-9a71-bb1cc1f85801") : secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:50.521157 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.520970    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/26658a95-087c-47ea-9a71-bb1cc1f85801-data-volume\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.521157 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.520977    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/26658a95-087c-47ea-9a71-bb1cc1f85801-crio-socket\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.532947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.532917    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dlvm\" (UniqueName: \"kubernetes.io/projected/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-api-access-6dlvm\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:50.607769 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.607737    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:50.621242 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.621215    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpjc2\" (UniqueName: \"kubernetes.io/projected/a98b8bc6-2242-4c67-abca-75e6b55c5d1f-kube-api-access-tpjc2\") pod \"migrator-6865c9967-8zkjz\" (UID: \"a98b8bc6-2242-4c67-abca-75e6b55c5d1f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz"
Jul 29 17:52:50.629916 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.629847    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpjc2\" (UniqueName: \"kubernetes.io/projected/a98b8bc6-2242-4c67-abca-75e6b55c5d1f-kube-api-access-tpjc2\") pod \"migrator-6865c9967-8zkjz\" (UID: \"a98b8bc6-2242-4c67-abca-75e6b55c5d1f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz"
Jul 29 17:52:50.668128 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.668095    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:50.698151 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.698123    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz"
Jul 29 17:52:50.918205 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.918179    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:50.960489 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:50.960456    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz"]
Jul 29 17:52:50.963591 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:50.963563    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98b8bc6_2242_4c67_abca_75e6b55c5d1f.slice/crio-0111645b1e7650f02e35d7b493f1206a4678f4ec17a58aca91b74a5193022053 WatchSource:0}: Error finding container 0111645b1e7650f02e35d7b493f1206a4678f4ec17a58aca91b74a5193022053: Status 404 returned error can't find the container with id 0111645b1e7650f02e35d7b493f1206a4678f4ec17a58aca91b74a5193022053
Jul 29 17:52:51.023378 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:51.023331    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:51.023536 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:51.023382    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:51.023536 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:51.023394    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:51.023536 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:51.023440    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 17:52:51.023536 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:51.023492    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-rbac-proxy-cm podName:26658a95-087c-47ea-9a71-bb1cc1f85801 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:52.023474638 +0000 UTC m=+41.531858227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hzmb7" (UID: "26658a95-087c-47ea-9a71-bb1cc1f85801") : configmap "kube-rbac-proxy" not found
Jul 29 17:52:51.023686 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:51.023548    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:51.023686 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:51.023581    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls podName:26658a95-087c-47ea-9a71-bb1cc1f85801 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:52.023573085 +0000 UTC m=+41.531956653 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hzmb7" (UID: "26658a95-087c-47ea-9a71-bb1cc1f85801") : secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:51.423079 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:51.423043    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:51.461102 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:51.461071    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz" event={"ID":"a98b8bc6-2242-4c67-abca-75e6b55c5d1f","Type":"ContainerStarted","Data":"0111645b1e7650f02e35d7b493f1206a4678f4ec17a58aca91b74a5193022053"}
Jul 29 17:52:51.462563 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:51.462540    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jhjsc" event={"ID":"37daa72f-68a5-462f-87f4-cbe6981c77a0","Type":"ContainerStarted","Data":"68935ace07e9573459dad39492b5e0d755d02f71741ab90f8aa893a724743eb0"}
Jul 29 17:52:51.478563 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:51.478527    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jhjsc" podStartSLOduration=3.296041151 podStartE2EDuration="6.478512773s" totalImagesPullingTime="3.182471622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:51.478255606 +0000 UTC m=+40.986639195" watchObservedRunningTime="2026-07-29 17:52:51.478512773 +0000 UTC m=+40.986896362"
Jul 29 17:52:51.742967 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:51.742889    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:51.842820 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:51.842790    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:52.016341 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.016267    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jf97c_3927a342-2b8b-47bd-bbb3-f78bbce3802a/dns-node-resolver/0.log"
Jul 29 17:52:52.032624 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.032603    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:52.032697 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.032632    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:52.032767 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:52.032754    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:52.032813 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:52.032802    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls podName:26658a95-087c-47ea-9a71-bb1cc1f85801 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:54.032790549 +0000 UTC m=+43.541174115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hzmb7" (UID: "26658a95-087c-47ea-9a71-bb1cc1f85801") : secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:52.033174 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.033151    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26658a95-087c-47ea-9a71-bb1cc1f85801-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:52.613047 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.613014    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-th4kf"]
Jul 29 17:52:52.629752 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.629715    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cttw7_eb4927fd-bc0a-4316-b43f-fb9144018eab/node-ca/0.log"
Jul 29 17:52:52.635278 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.635256    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-th4kf"
Jul 29 17:52:52.643736 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.642577    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 29 17:52:52.643736 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.643266    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 29 17:52:52.644536 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.643895    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 29 17:52:52.644687 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.644668    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 29 17:52:52.644751 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.644738    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-pznmq\""
Jul 29 17:52:52.647991 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.647953    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-th4kf"]
Jul 29 17:52:52.704038 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.703997    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:52.736339 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.736303    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6b01b8bd-b204-4824-8e5c-5508061e9e5a-signing-key\") pod \"service-ca-658d7b7c78-th4kf\" (UID: \"6b01b8bd-b204-4824-8e5c-5508061e9e5a\") " pod="openshift-service-ca/service-ca-658d7b7c78-th4kf"
Jul 29 17:52:52.736509 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.736426    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6b01b8bd-b204-4824-8e5c-5508061e9e5a-signing-cabundle\") pod \"service-ca-658d7b7c78-th4kf\" (UID: \"6b01b8bd-b204-4824-8e5c-5508061e9e5a\") " pod="openshift-service-ca/service-ca-658d7b7c78-th4kf"
Jul 29 17:52:52.736509 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.736459    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwl4m\" (UniqueName: \"kubernetes.io/projected/6b01b8bd-b204-4824-8e5c-5508061e9e5a-kube-api-access-bwl4m\") pod \"service-ca-658d7b7c78-th4kf\" (UID: \"6b01b8bd-b204-4824-8e5c-5508061e9e5a\") " pod="openshift-service-ca/service-ca-658d7b7c78-th4kf"
Jul 29 17:52:52.837030 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.836991    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6b01b8bd-b204-4824-8e5c-5508061e9e5a-signing-key\") pod \"service-ca-658d7b7c78-th4kf\" (UID: \"6b01b8bd-b204-4824-8e5c-5508061e9e5a\") " pod="openshift-service-ca/service-ca-658d7b7c78-th4kf"
Jul 29 17:52:52.837209 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.837088    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6b01b8bd-b204-4824-8e5c-5508061e9e5a-signing-cabundle\") pod \"service-ca-658d7b7c78-th4kf\" (UID: \"6b01b8bd-b204-4824-8e5c-5508061e9e5a\") " pod="openshift-service-ca/service-ca-658d7b7c78-th4kf"
Jul 29 17:52:52.837209 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.837133    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwl4m\" (UniqueName: \"kubernetes.io/projected/6b01b8bd-b204-4824-8e5c-5508061e9e5a-kube-api-access-bwl4m\") pod \"service-ca-658d7b7c78-th4kf\" (UID: \"6b01b8bd-b204-4824-8e5c-5508061e9e5a\") " pod="openshift-service-ca/service-ca-658d7b7c78-th4kf"
Jul 29 17:52:52.837717 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.837695    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6b01b8bd-b204-4824-8e5c-5508061e9e5a-signing-cabundle\") pod \"service-ca-658d7b7c78-th4kf\" (UID: \"6b01b8bd-b204-4824-8e5c-5508061e9e5a\") " pod="openshift-service-ca/service-ca-658d7b7c78-th4kf"
Jul 29 17:52:52.839282 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.839262    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6b01b8bd-b204-4824-8e5c-5508061e9e5a-signing-key\") pod \"service-ca-658d7b7c78-th4kf\" (UID: \"6b01b8bd-b204-4824-8e5c-5508061e9e5a\") " pod="openshift-service-ca/service-ca-658d7b7c78-th4kf"
Jul 29 17:52:52.855861 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.855838    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwl4m\" (UniqueName: \"kubernetes.io/projected/6b01b8bd-b204-4824-8e5c-5508061e9e5a-kube-api-access-bwl4m\") pod \"service-ca-658d7b7c78-th4kf\" (UID: \"6b01b8bd-b204-4824-8e5c-5508061e9e5a\") " pod="openshift-service-ca/service-ca-658d7b7c78-th4kf"
Jul 29 17:52:52.908734 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.908703    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:52.948598 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.948557    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-th4kf"
Jul 29 17:52:52.998093 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:52.998059    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:53.081972 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:53.081938    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-th4kf"]
Jul 29 17:52:53.087546 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:53.087523    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:53.161967 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:52:53.161889    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b01b8bd_b204_4824_8e5c_5508061e9e5a.slice/crio-495f42d6eaf1294d725239bcd97c170606a6948ba2c48d58222309cd1fe99e20 WatchSource:0}: Error finding container 495f42d6eaf1294d725239bcd97c170606a6948ba2c48d58222309cd1fe99e20: Status 404 returned error can't find the container with id 495f42d6eaf1294d725239bcd97c170606a6948ba2c48d58222309cd1fe99e20
Jul 29 17:52:53.468187 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:53.468152    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz" event={"ID":"a98b8bc6-2242-4c67-abca-75e6b55c5d1f","Type":"ContainerStarted","Data":"16c24cd61227851775d87ef56e264ab4168736ae44aa48a87099ae34bb95f5db"}
Jul 29 17:52:53.468324 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:53.468212    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz" event={"ID":"a98b8bc6-2242-4c67-abca-75e6b55c5d1f","Type":"ContainerStarted","Data":"51135aabd35e575e1ed55a3d828c79573697fd0282aeaebb2bb798780187e0f2"}
Jul 29 17:52:53.469219 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:53.469198    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-th4kf" event={"ID":"6b01b8bd-b204-4824-8e5c-5508061e9e5a","Type":"ContainerStarted","Data":"495f42d6eaf1294d725239bcd97c170606a6948ba2c48d58222309cd1fe99e20"}
Jul 29 17:52:53.617532 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:53.617504    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jhjsc_37daa72f-68a5-462f-87f4-cbe6981c77a0/serve-healthcheck-canary/0.log"
Jul 29 17:52:53.643714 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:53.643685    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:52:53.643865 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:53.643724    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:52:53.643865 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:53.643823    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 17:52:53.643865 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:53.643839    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f78b6ccd-vj58l: secret "image-registry-tls" not found
Jul 29 17:52:53.643981 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:53.643879    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 17:52:53.643981 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:53.643888    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls podName:2a4db46e-c7d3-4b21-935e-b2e79d07c5cf nodeName:}" failed. No retries permitted until 2026-07-29 17:53:01.643873391 +0000 UTC m=+51.152256957 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls") pod "image-registry-67f78b6ccd-vj58l" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf") : secret "image-registry-tls" not found
Jul 29 17:52:53.643981 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:53.643921    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls podName:dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e nodeName:}" failed. No retries permitted until 2026-07-29 17:53:01.643907811 +0000 UTC m=+51.152291378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls") pod "dns-default-8v82s" (UID: "dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e") : secret "dns-default-metrics-tls" not found
Jul 29 17:52:53.851444 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:53.851346    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:54.047520 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:54.047480    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:54.047686 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:54.047629    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:54.047747 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:54.047694    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls podName:26658a95-087c-47ea-9a71-bb1cc1f85801 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:58.047673564 +0000 UTC m=+47.556057150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hzmb7" (UID: "26658a95-087c-47ea-9a71-bb1cc1f85801") : secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:56.476895 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:56.476858    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-th4kf" event={"ID":"6b01b8bd-b204-4824-8e5c-5508061e9e5a","Type":"ContainerStarted","Data":"8ccd56141821c43ba966c1e9ffb5b8b0c54dac85932c9683324f0dad886b3973"}
Jul 29 17:52:56.497823 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:56.497777    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8zkjz" podStartSLOduration=4.27126718 podStartE2EDuration="6.497762036s" totalImagesPullingTime="2.226494856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:53.492899508 +0000 UTC m=+43.001283099" watchObservedRunningTime="2026-07-29 17:52:56.497762036 +0000 UTC m=+46.006145624"
Jul 29 17:52:56.498014 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:56.497995    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-th4kf" podStartSLOduration=2.053249604 podStartE2EDuration="4.497990402s" totalImagesPullingTime="2.444740798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:56.497893422 +0000 UTC m=+46.006276996" watchObservedRunningTime="2026-07-29 17:52:56.497990402 +0000 UTC m=+46.006374031"
Jul 29 17:52:56.976914 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:56.976855    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:57.077356 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:57.077321    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:57.213760 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:57.213721    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:57.314073 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:57.313982    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:58.074899 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:58.074864    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:58.080045 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:58.080020    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:52:58.080189 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:58.080171    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:58.080254 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:52:58.080247    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls podName:26658a95-087c-47ea-9a71-bb1cc1f85801 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:06.080225754 +0000 UTC m=+55.588609334 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hzmb7" (UID: "26658a95-087c-47ea-9a71-bb1cc1f85801") : secret "insights-runtime-extractor-tls" not found
Jul 29 17:52:58.501720 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:58.501688    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:58.857623 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:58.857549    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:59.082345 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:59.082310    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:52:59.542091 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:52:59.542057    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:00.051118 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:00.051082    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:00.564506 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:00.564472    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:00.664884 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:00.664851    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:00.774197 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:00.774166    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:00.877684 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:00.877651    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:01.267286 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:01.267201    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:01.445853 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:01.445814    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:01.509218 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:01.509181    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:01.707468 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:01.707435    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:53:01.707849 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:01.707476    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:53:01.709827 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:01.709793    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e-metrics-tls\") pod \"dns-default-8v82s\" (UID: \"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e\") " pod="openshift-dns/dns-default-8v82s"
Jul 29 17:53:01.709966 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:01.709850    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls\") pod \"image-registry-67f78b6ccd-vj58l\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") " pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:53:01.795425 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:01.795390    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:53:01.812793 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:01.812767    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8v82s"
Jul 29 17:53:01.957350 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:01.957314    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67f78b6ccd-vj58l"]
Jul 29 17:53:01.961615 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:53:01.961585    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a4db46e_c7d3_4b21_935e_b2e79d07c5cf.slice/crio-5ed0f7c528bb13e61aeaa1d83629e64f993f35395c6ea0dc3ac0fe19ad734102 WatchSource:0}: Error finding container 5ed0f7c528bb13e61aeaa1d83629e64f993f35395c6ea0dc3ac0fe19ad734102: Status 404 returned error can't find the container with id 5ed0f7c528bb13e61aeaa1d83629e64f993f35395c6ea0dc3ac0fe19ad734102
Jul 29 17:53:01.966908 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:01.966864    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8v82s"]
Jul 29 17:53:01.970111 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:53:01.970090    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcc6927f_2ddf_4b0a_a52f_24fbfc2aad7e.slice/crio-02e1ecb4691e7139001340aa1a8051f8adb24ce98141321c40c93f582160eef9 WatchSource:0}: Error finding container 02e1ecb4691e7139001340aa1a8051f8adb24ce98141321c40c93f582160eef9: Status 404 returned error can't find the container with id 02e1ecb4691e7139001340aa1a8051f8adb24ce98141321c40c93f582160eef9
Jul 29 17:53:02.049582 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:02.049551    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:02.449450 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:02.449419    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:02.493565 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:02.493530    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8v82s" event={"ID":"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e","Type":"ContainerStarted","Data":"02e1ecb4691e7139001340aa1a8051f8adb24ce98141321c40c93f582160eef9"}
Jul 29 17:53:02.495100 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:02.495072    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l" event={"ID":"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf","Type":"ContainerStarted","Data":"db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702"}
Jul 29 17:53:02.495223 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:02.495117    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l" event={"ID":"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf","Type":"ContainerStarted","Data":"5ed0f7c528bb13e61aeaa1d83629e64f993f35395c6ea0dc3ac0fe19ad734102"}
Jul 29 17:53:02.495223 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:02.495201    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:53:02.519910 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:02.519857    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l" podStartSLOduration=21.519836528 podStartE2EDuration="21.519836528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:02.519145181 +0000 UTC m=+52.027528769" watchObservedRunningTime="2026-07-29 17:53:02.519836528 +0000 UTC m=+52.028220117"
Jul 29 17:53:02.637204 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:02.637168    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:03.053894 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:03.053856    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:03.153985 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:03.153956    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:03.639388 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:03.639344    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:04.244890 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:04.244852    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:04.435960 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:04.435929    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:04.834980 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:04.834893    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:04.958091 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:04.958057    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:05.052328 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:05.052290    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:05.442337 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:05.442303    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:05.506013 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:05.505977    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8v82s" event={"ID":"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e","Type":"ContainerStarted","Data":"335daf0f167da32beeec37081fdb62d2182656c039dd24d20696fb6fde924437"}
Jul 29 17:53:05.506168 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:05.506036    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8v82s" event={"ID":"dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e","Type":"ContainerStarted","Data":"894ccd59561a24db2737c30f1ddb1739850ef6bb63f704662734b7dd25fe5b9f"}
Jul 29 17:53:05.506168 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:05.506117    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8v82s"
Jul 29 17:53:05.524453 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:05.524408    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8v82s" podStartSLOduration=18.0545589 podStartE2EDuration="20.524393032s" totalImagesPullingTime="2.469834132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:05.523701852 +0000 UTC m=+55.032085444" watchObservedRunningTime="2026-07-29 17:53:05.524393032 +0000 UTC m=+55.032776597"
Jul 29 17:53:06.144348 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:06.144302    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:53:06.146792 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:06.146765    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26658a95-087c-47ea-9a71-bb1cc1f85801-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hzmb7\" (UID: \"26658a95-087c-47ea-9a71-bb1cc1f85801\") " pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:53:06.170710 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:06.170682    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hzmb7"
Jul 29 17:53:06.239737 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:06.239704    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:06.295528 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:06.295498    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hzmb7"]
Jul 29 17:53:06.298607 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:53:06.298580    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26658a95_087c_47ea_9a71_bb1cc1f85801.slice/crio-c40a67805ead34bc201dc955493ecc42325ebf2f0d8797a8c6b55f098c930e53 WatchSource:0}: Error finding container c40a67805ead34bc201dc955493ecc42325ebf2f0d8797a8c6b55f098c930e53: Status 404 returned error can't find the container with id c40a67805ead34bc201dc955493ecc42325ebf2f0d8797a8c6b55f098c930e53
Jul 29 17:53:06.446801 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:06.446718    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:06.510532 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:06.510498    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hzmb7" event={"ID":"26658a95-087c-47ea-9a71-bb1cc1f85801","Type":"ContainerStarted","Data":"c40a67805ead34bc201dc955493ecc42325ebf2f0d8797a8c6b55f098c930e53"}
Jul 29 17:53:06.836954 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:06.836873    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:07.037631 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:07.037536    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:07.440726 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:07.440689    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:07.660440 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:07.660405    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:11.528717 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:11.528683    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hzmb7" event={"ID":"26658a95-087c-47ea-9a71-bb1cc1f85801","Type":"ContainerStarted","Data":"6e11460f132625d0c403da222a8a16783f8d517875cf4969d552947b58cee8cb"}
Jul 29 17:53:12.443896 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:12.443794    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9wvx"
Jul 29 17:53:15.513092 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.513060    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8v82s"
Jul 29 17:53:15.544904 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.544867    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hzmb7" event={"ID":"26658a95-087c-47ea-9a71-bb1cc1f85801","Type":"ContainerStarted","Data":"ad79aefa4ddaa77c2d8038d4403700cfe01594d1ef649bc8843c945fa53b6208"}
Jul 29 17:53:15.811947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.811854    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:53:15.811947 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.811929    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:53:15.814624 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.814595    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 17:53:15.815162 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.815137    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 17:53:15.824883 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.824860    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a962e36-54ac-40e3-8c51-c317c81f2133-original-pull-secret\") pod \"global-pull-secret-syncer-5qf55\" (UID: \"9a962e36-54ac-40e3-8c51-c317c81f2133\") " pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:53:15.824977 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.824907    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad618126-a7f9-40d0-a92e-4b80729d8384-metrics-certs\") pod \"network-metrics-daemon-vcnql\" (UID: \"ad618126-a7f9-40d0-a92e-4b80729d8384\") " pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:53:15.912449 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.912417    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jnc\" (UniqueName: \"kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc\") pod \"network-check-target-pv5rx\" (UID: \"ced120c9-89f5-45bb-85b8-37d7e0b82f9e\") " pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:53:15.915019 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.914989    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 17:53:15.926135 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.926110    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 17:53:15.936936 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:15.936910    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49jnc\" (UniqueName: \"kubernetes.io/projected/ced120c9-89f5-45bb-85b8-37d7e0b82f9e-kube-api-access-49jnc\") pod \"network-check-target-pv5rx\" (UID: \"ced120c9-89f5-45bb-85b8-37d7e0b82f9e\") " pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:53:16.087798 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.087724    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-llfsb\""
Jul 29 17:53:16.095918 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.095892    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:53:16.111745 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.111710    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w7s6p\""
Jul 29 17:53:16.113224 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.112984    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qf55"
Jul 29 17:53:16.119618 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.119598    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vcnql"
Jul 29 17:53:16.259667 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:53:16.259639    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced120c9_89f5_45bb_85b8_37d7e0b82f9e.slice/crio-19de20a00ac16325108019a51ccb90bb8a953729dcf0d6929918558c4738fda2 WatchSource:0}: Error finding container 19de20a00ac16325108019a51ccb90bb8a953729dcf0d6929918558c4738fda2: Status 404 returned error can't find the container with id 19de20a00ac16325108019a51ccb90bb8a953729dcf0d6929918558c4738fda2
Jul 29 17:53:16.259907 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.259882    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pv5rx"]
Jul 29 17:53:16.285332 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.285291    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5qf55"]
Jul 29 17:53:16.288599 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:53:16.288569    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a962e36_54ac_40e3_8c51_c317c81f2133.slice/crio-fe70f944f9f31f46a904e3deaa49051beffb36c8709a5eb2e32b5f0e2ac1483a WatchSource:0}: Error finding container fe70f944f9f31f46a904e3deaa49051beffb36c8709a5eb2e32b5f0e2ac1483a: Status 404 returned error can't find the container with id fe70f944f9f31f46a904e3deaa49051beffb36c8709a5eb2e32b5f0e2ac1483a
Jul 29 17:53:16.304126 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.304078    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vcnql"]
Jul 29 17:53:16.306667 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:53:16.306541    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad618126_a7f9_40d0_a92e_4b80729d8384.slice/crio-b9ea0c951874f9c68b989a6359a786f0bd6ab284ee2478d32e32752be4ebd3a5 WatchSource:0}: Error finding container b9ea0c951874f9c68b989a6359a786f0bd6ab284ee2478d32e32752be4ebd3a5: Status 404 returned error can't find the container with id b9ea0c951874f9c68b989a6359a786f0bd6ab284ee2478d32e32752be4ebd3a5
Jul 29 17:53:16.550139 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.550100    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5qf55" event={"ID":"9a962e36-54ac-40e3-8c51-c317c81f2133","Type":"ContainerStarted","Data":"fe70f944f9f31f46a904e3deaa49051beffb36c8709a5eb2e32b5f0e2ac1483a"}
Jul 29 17:53:16.551462 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.551429    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vcnql" event={"ID":"ad618126-a7f9-40d0-a92e-4b80729d8384","Type":"ContainerStarted","Data":"b9ea0c951874f9c68b989a6359a786f0bd6ab284ee2478d32e32752be4ebd3a5"}
Jul 29 17:53:16.552762 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.552736    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pv5rx" event={"ID":"ced120c9-89f5-45bb-85b8-37d7e0b82f9e","Type":"ContainerStarted","Data":"19de20a00ac16325108019a51ccb90bb8a953729dcf0d6929918558c4738fda2"}
Jul 29 17:53:16.771257 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.770738    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"]
Jul 29 17:53:16.773919 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.773883    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"]
Jul 29 17:53:16.774061 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.774017    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:16.777662 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.777025    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 17:53:16.777662 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.777038    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 17:53:16.777662 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.777200    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 29 17:53:16.777662 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.777403    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 17:53:16.777986 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.777745    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:16.778395 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.778061    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.782962 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.782940    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 29 17:53:16.783195 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.783179    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 29 17:53:16.783432 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.783414    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 29 17:53:16.783796 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.783614    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 29 17:53:16.791829 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.791807    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"]
Jul 29 17:53:16.793681 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.793658    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"]
Jul 29 17:53:16.801808 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.801211    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67f78b6ccd-vj58l"]
Jul 29 17:53:16.818968 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.818597    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.818968 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.818661    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/59f3badb-4466-46c8-a72a-f59120910dfa-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.818968 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.818689    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3062cf4-3f71-4b02-b70c-a568c7ba76a2-tmp\") pod \"klusterlet-addon-workmgr-dcf9b6bf6-j59hs\" (UID: \"a3062cf4-3f71-4b02-b70c-a568c7ba76a2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:16.818968 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.818711    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-ca\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.818968 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.818739    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.818968 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.818768    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxttn\" (UniqueName: \"kubernetes.io/projected/a3062cf4-3f71-4b02-b70c-a568c7ba76a2-kube-api-access-nxttn\") pod \"klusterlet-addon-workmgr-dcf9b6bf6-j59hs\" (UID: \"a3062cf4-3f71-4b02-b70c-a568c7ba76a2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:16.818968 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.818793    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3062cf4-3f71-4b02-b70c-a568c7ba76a2-klusterlet-config\") pod \"klusterlet-addon-workmgr-dcf9b6bf6-j59hs\" (UID: \"a3062cf4-3f71-4b02-b70c-a568c7ba76a2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:16.818968 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.818832    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9zsv\" (UniqueName: \"kubernetes.io/projected/59f3badb-4466-46c8-a72a-f59120910dfa-kube-api-access-k9zsv\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.818968 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.818856    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-hub\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.887973 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.887894    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:16.897024 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.896973    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84f857b7cd-rfs4d"]
Jul 29 17:53:16.900403 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.900380    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:16.919402 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919344    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b62d4027-7580-4b8a-a699-f90c6b318f59-image-registry-private-configuration\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:16.919525 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919434    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b62d4027-7580-4b8a-a699-f90c6b318f59-registry-tls\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:16.919525 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919481    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b62d4027-7580-4b8a-a699-f90c6b318f59-trusted-ca\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:16.919525 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919514    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/59f3badb-4466-46c8-a72a-f59120910dfa-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.919630 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919540    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3062cf4-3f71-4b02-b70c-a568c7ba76a2-tmp\") pod \"klusterlet-addon-workmgr-dcf9b6bf6-j59hs\" (UID: \"a3062cf4-3f71-4b02-b70c-a568c7ba76a2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:16.919630 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919583    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-ca\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.919630 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919611    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.919718 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919656    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b62d4027-7580-4b8a-a699-f90c6b318f59-installation-pull-secrets\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:16.919718 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919687    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxttn\" (UniqueName: \"kubernetes.io/projected/a3062cf4-3f71-4b02-b70c-a568c7ba76a2-kube-api-access-nxttn\") pod \"klusterlet-addon-workmgr-dcf9b6bf6-j59hs\" (UID: \"a3062cf4-3f71-4b02-b70c-a568c7ba76a2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:16.919789 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919732    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3062cf4-3f71-4b02-b70c-a568c7ba76a2-klusterlet-config\") pod \"klusterlet-addon-workmgr-dcf9b6bf6-j59hs\" (UID: \"a3062cf4-3f71-4b02-b70c-a568c7ba76a2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:16.919843 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919761    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b62d4027-7580-4b8a-a699-f90c6b318f59-ca-trust-extracted\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:16.919843 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919831    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ksgr\" (UniqueName: \"kubernetes.io/projected/b62d4027-7580-4b8a-a699-f90c6b318f59-kube-api-access-8ksgr\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:16.919952 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919873    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9zsv\" (UniqueName: \"kubernetes.io/projected/59f3badb-4466-46c8-a72a-f59120910dfa-kube-api-access-k9zsv\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.919952 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919900    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-hub\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.919952 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919938    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b62d4027-7580-4b8a-a699-f90c6b318f59-bound-sa-token\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:16.920093 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.919967    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b62d4027-7580-4b8a-a699-f90c6b318f59-registry-certificates\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:16.920093 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.920005    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.921091 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.921067    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/59f3badb-4466-46c8-a72a-f59120910dfa-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.921437 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.921414    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3062cf4-3f71-4b02-b70c-a568c7ba76a2-tmp\") pod \"klusterlet-addon-workmgr-dcf9b6bf6-j59hs\" (UID: \"a3062cf4-3f71-4b02-b70c-a568c7ba76a2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:16.926015 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.925968    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-ca\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.926640 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.926162    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a3062cf4-3f71-4b02-b70c-a568c7ba76a2-klusterlet-config\") pod \"klusterlet-addon-workmgr-dcf9b6bf6-j59hs\" (UID: \"a3062cf4-3f71-4b02-b70c-a568c7ba76a2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:16.926640 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.926602    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.927378 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.927322    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-hub\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.931020 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.930977    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/59f3badb-4466-46c8-a72a-f59120910dfa-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:16.934175 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.934155    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84f857b7cd-rfs4d"]
Jul 29 17:53:16.944148 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.944120    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxttn\" (UniqueName: \"kubernetes.io/projected/a3062cf4-3f71-4b02-b70c-a568c7ba76a2-kube-api-access-nxttn\") pod \"klusterlet-addon-workmgr-dcf9b6bf6-j59hs\" (UID: \"a3062cf4-3f71-4b02-b70c-a568c7ba76a2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:16.945780 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:16.945733    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9zsv\" (UniqueName: \"kubernetes.io/projected/59f3badb-4466-46c8-a72a-f59120910dfa-kube-api-access-k9zsv\") pod \"cluster-proxy-proxy-agent-55899649c8-8n9qz\" (UID: \"59f3badb-4466-46c8-a72a-f59120910dfa\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:17.006211 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.006173    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:17.021085 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.021052    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b62d4027-7580-4b8a-a699-f90c6b318f59-bound-sa-token\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.021244 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.021104    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b62d4027-7580-4b8a-a699-f90c6b318f59-registry-certificates\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.021244 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.021147    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b62d4027-7580-4b8a-a699-f90c6b318f59-image-registry-private-configuration\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.021244 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.021189    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b62d4027-7580-4b8a-a699-f90c6b318f59-registry-tls\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.021244 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.021218    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b62d4027-7580-4b8a-a699-f90c6b318f59-trusted-ca\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.021514 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.021255    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b62d4027-7580-4b8a-a699-f90c6b318f59-installation-pull-secrets\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.021514 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.021289    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b62d4027-7580-4b8a-a699-f90c6b318f59-ca-trust-extracted\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.021635 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.021610    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ksgr\" (UniqueName: \"kubernetes.io/projected/b62d4027-7580-4b8a-a699-f90c6b318f59-kube-api-access-8ksgr\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.021693 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.021667    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b62d4027-7580-4b8a-a699-f90c6b318f59-ca-trust-extracted\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.023284 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.023236    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b62d4027-7580-4b8a-a699-f90c6b318f59-trusted-ca\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.026681 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.026639    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b62d4027-7580-4b8a-a699-f90c6b318f59-registry-tls\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.026917 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.026894    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b62d4027-7580-4b8a-a699-f90c6b318f59-registry-certificates\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.027603 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.027556    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b62d4027-7580-4b8a-a699-f90c6b318f59-image-registry-private-configuration\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.030520 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.029967    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b62d4027-7580-4b8a-a699-f90c6b318f59-installation-pull-secrets\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.034107 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.034084    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b62d4027-7580-4b8a-a699-f90c6b318f59-bound-sa-token\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.036194 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.036173    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ksgr\" (UniqueName: \"kubernetes.io/projected/b62d4027-7580-4b8a-a699-f90c6b318f59-kube-api-access-8ksgr\") pod \"image-registry-84f857b7cd-rfs4d\" (UID: \"b62d4027-7580-4b8a-a699-f90c6b318f59\") " pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:17.082808 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.082737    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:17.088831 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.088571    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:17.101876 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.101702    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"
Jul 29 17:53:17.182038 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.181954    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:17.214353 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:17.213976    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:19.139811 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:19.139768    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:19.245021 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:19.244992    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:19.377965 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:19.377880    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:19.694489 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:19.694431    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:21.747314 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:21.747279    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:21.855812 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:21.855780    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:22.549309 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.549217    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz"]
Jul 29 17:53:22.549309 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.549294    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84f857b7cd-rfs4d"]
Jul 29 17:53:22.556666 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:53:22.556622    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59f3badb_4466_46c8_a72a_f59120910dfa.slice/crio-595d67819758e9b5ce96c61bc6e14030c803a9cbe28900d8ac860d23effe4dff WatchSource:0}: Error finding container 595d67819758e9b5ce96c61bc6e14030c803a9cbe28900d8ac860d23effe4dff: Status 404 returned error can't find the container with id 595d67819758e9b5ce96c61bc6e14030c803a9cbe28900d8ac860d23effe4dff
Jul 29 17:53:22.557336 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:53:22.557309    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb62d4027_7580_4b8a_a699_f90c6b318f59.slice/crio-44227e45bb76819fed618f6f0ef70fdff0a4e0d9b098f1573172590f1b58d918 WatchSource:0}: Error finding container 44227e45bb76819fed618f6f0ef70fdff0a4e0d9b098f1573172590f1b58d918: Status 404 returned error can't find the container with id 44227e45bb76819fed618f6f0ef70fdff0a4e0d9b098f1573172590f1b58d918
Jul 29 17:53:22.578602 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.578566    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hzmb7" event={"ID":"26658a95-087c-47ea-9a71-bb1cc1f85801","Type":"ContainerStarted","Data":"1b8f8c085ddcd4cb030a6efabbb58aef0cb72afa5f9c8e619fb551d9caa8e6d1"}
Jul 29 17:53:22.581195 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.581123    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5qf55" event={"ID":"9a962e36-54ac-40e3-8c51-c317c81f2133","Type":"ContainerStarted","Data":"d398a1d1f2a3d0c7ab1390ed4331952e4e72e9702d695ef53262a0fd29de1056"}
Jul 29 17:53:22.582628 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.582600    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz" event={"ID":"59f3badb-4466-46c8-a72a-f59120910dfa","Type":"ContainerStarted","Data":"595d67819758e9b5ce96c61bc6e14030c803a9cbe28900d8ac860d23effe4dff"}
Jul 29 17:53:22.585330 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.585277    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"]
Jul 29 17:53:22.588745 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.587061    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vcnql" event={"ID":"ad618126-a7f9-40d0-a92e-4b80729d8384","Type":"ContainerStarted","Data":"83d1b2e1ac69fc5752dd85e06bf9feae0e7d74328e5e140956d2d5090fda7405"}
Jul 29 17:53:22.591434 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.591395    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pv5rx" event={"ID":"ced120c9-89f5-45bb-85b8-37d7e0b82f9e","Type":"ContainerStarted","Data":"28d73839c5188fa31fc17aec8e26bf2bc281d52cf9100af66a405135322cc38f"}
Jul 29 17:53:22.591595 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.591577    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:53:22.594273 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.594239    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d" event={"ID":"b62d4027-7580-4b8a-a699-f90c6b318f59","Type":"ContainerStarted","Data":"44227e45bb76819fed618f6f0ef70fdff0a4e0d9b098f1573172590f1b58d918"}
Jul 29 17:53:22.606313 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.606246    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hzmb7" podStartSLOduration=16.699761563 podStartE2EDuration="32.606220791s" totalImagesPullingTime="15.906459228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:22.60602309 +0000 UTC m=+72.114406717" watchObservedRunningTime="2026-07-29 17:53:22.606220791 +0000 UTC m=+72.114604380"
Jul 29 17:53:22.639477 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.639423    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5qf55" podStartSLOduration=64.594127927 podStartE2EDuration="1m10.639398906s" totalImagesPullingTime="6.045270979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:22.638181296 +0000 UTC m=+72.146564884" watchObservedRunningTime="2026-07-29 17:53:22.639398906 +0000 UTC m=+72.147782499"
Jul 29 17:53:22.667056 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.667001    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pv5rx" podStartSLOduration=64.60789833 podStartE2EDuration="1m10.666981173s" totalImagesPullingTime="6.059082843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:22.666236371 +0000 UTC m=+72.174619959" watchObservedRunningTime="2026-07-29 17:53:22.666981173 +0000 UTC m=+72.175364763"
Jul 29 17:53:22.723889 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.723860    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:22.820419 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.820341    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:22.889235 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:22.889204    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:23.603190 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:23.603149    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vcnql" event={"ID":"ad618126-a7f9-40d0-a92e-4b80729d8384","Type":"ContainerStarted","Data":"ca178f69831dc240a3894574cfa1a06e22a58c67316b20a7d57f3038be87fbe8"}
Jul 29 17:53:23.605263 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:23.605231    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d" event={"ID":"b62d4027-7580-4b8a-a699-f90c6b318f59","Type":"ContainerStarted","Data":"17ce1a410b2f22b2ab4334c34fdf0603fc095182b89949e0a7570592903652cd"}
Jul 29 17:53:23.605421 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:23.605300    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:23.607442 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:23.607414    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs" event={"ID":"a3062cf4-3f71-4b02-b70c-a568c7ba76a2","Type":"ContainerStarted","Data":"c8984c42833d4707f65fa5db6bd63a8219c00a9b65bad089f8f0052ceee83b9d"}
Jul 29 17:53:23.621628 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:23.620966    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vcnql" podStartSLOduration=65.60832527 podStartE2EDuration="1m11.620948433s" totalImagesPullingTime="6.012623163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:23.620881438 +0000 UTC m=+73.129265026" watchObservedRunningTime="2026-07-29 17:53:23.620948433 +0000 UTC m=+73.129332018"
Jul 29 17:53:23.644356 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:23.643130    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d" podStartSLOduration=7.643111809 podStartE2EDuration="7.643111809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:23.642693298 +0000 UTC m=+73.151076898" watchObservedRunningTime="2026-07-29 17:53:23.643111809 +0000 UTC m=+73.151495398"
Jul 29 17:53:23.746837 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:23.746802    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:23.809095 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:23.809015    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:23.871844 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:23.871680    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:24.454657 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:24.454572    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:24.525906 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:24.525831    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:24.825254 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:24.825178    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:25.051452 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:25.051420    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:25.151210 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:25.151144    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:25.252135 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:25.252104    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:26.807514 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:26.807479    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:53:28.628634 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:28.628597    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs" event={"ID":"a3062cf4-3f71-4b02-b70c-a568c7ba76a2","Type":"ContainerStarted","Data":"8d58faa46c07b8994905434690e05d364ca03c85deae064a74a88419f74d372a"}
Jul 29 17:53:28.629097 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:28.628757    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:28.630789 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:28.630763    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz" event={"ID":"59f3badb-4466-46c8-a72a-f59120910dfa","Type":"ContainerStarted","Data":"8f5580500cecc8d964cbbf43e7b4627cad934590a2c035c4e937bde0805e785b"}
Jul 29 17:53:28.630906 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:28.630829    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs"
Jul 29 17:53:28.630906 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:28.630848    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz" event={"ID":"59f3badb-4466-46c8-a72a-f59120910dfa","Type":"ContainerStarted","Data":"27cc90042af9f0438fe525beef7b5991e6643e89d29f36cc568ba8af03df5af4"}
Jul 29 17:53:28.630906 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:28.630860    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz" event={"ID":"59f3badb-4466-46c8-a72a-f59120910dfa","Type":"ContainerStarted","Data":"9297d935e9eecba6b8a0ad71e9d3339483c7258d40d513081f8e2994c75067dd"}
Jul 29 17:53:28.646330 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:28.646260    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dcf9b6bf6-j59hs" podStartSLOduration=7.402137685 podStartE2EDuration="12.646246154s" totalImagesPullingTime="5.244108469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:28.645531226 +0000 UTC m=+78.153914813" watchObservedRunningTime="2026-07-29 17:53:28.646246154 +0000 UTC m=+78.154629741"
Jul 29 17:53:28.668315 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:28.668272    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55899649c8-8n9qz" podStartSLOduration=7.388548473 podStartE2EDuration="12.66825802s" totalImagesPullingTime="5.279709547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:28.667070862 +0000 UTC m=+78.175454450" watchObservedRunningTime="2026-07-29 17:53:28.66825802 +0000 UTC m=+78.176641608"
Jul 29 17:53:29.476896 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:29.476865    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:29.574815 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:29.574785    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:29.659835 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:29.659805    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:29.756653 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:29.756548    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:29.815014 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:29.814987    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:30.215339 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:30.215304    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:30.277279 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:30.277249    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:31.012409 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:31.012359    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:31.849874 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:31.849820    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:31.994797 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:31.994760    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:34.792385 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:34.792226    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:34.846901 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:34.846865    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:35.532693 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.532663    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:35.759412 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.759344    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:35.826000 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.825913    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nxhrj"]
Jul 29 17:53:35.830747 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.830726    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.833390 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.833356    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lwjtf\""
Jul 29 17:53:35.834576 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.834392    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 17:53:35.834576 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.834414    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 17:53:35.834576 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.834439    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 17:53:35.834576 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.834480    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 17:53:35.834917 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.834683    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 17:53:35.834917 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.834881    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 17:53:35.853121 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.853101    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:35.868078 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.868052    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-wtmp\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.868187 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.868082    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.868187 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.868108    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z5ql\" (UniqueName: \"kubernetes.io/projected/af725a92-e993-462d-a9e6-99f1298b08c0-kube-api-access-6z5ql\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.868301 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.868185    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-textfile\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.868301 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.868222    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/af725a92-e993-462d-a9e6-99f1298b08c0-root\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.868431 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.868299    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.868431 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.868337    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af725a92-e993-462d-a9e6-99f1298b08c0-sys\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.868431 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.868383    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af725a92-e993-462d-a9e6-99f1298b08c0-metrics-client-ca\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.868431 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.868428    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-tls\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.947499 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.947467    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:35.969506 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969476    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-textfile\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969506 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969508    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/af725a92-e993-462d-a9e6-99f1298b08c0-root\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969672 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969545    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969672 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969578    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af725a92-e993-462d-a9e6-99f1298b08c0-sys\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969672 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969598    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af725a92-e993-462d-a9e6-99f1298b08c0-metrics-client-ca\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969672 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969602    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/af725a92-e993-462d-a9e6-99f1298b08c0-root\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969672 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969649    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-tls\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969886 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969667    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af725a92-e993-462d-a9e6-99f1298b08c0-sys\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969886 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969677    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-wtmp\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969886 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969705    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969886 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969734    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z5ql\" (UniqueName: \"kubernetes.io/projected/af725a92-e993-462d-a9e6-99f1298b08c0-kube-api-access-6z5ql\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969886 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969805    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-wtmp\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.969886 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.969883    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-textfile\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.970255 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.970233    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af725a92-e993-462d-a9e6-99f1298b08c0-metrics-client-ca\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.970342 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.970316    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.971912 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.971892    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.972009 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.971973    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/af725a92-e993-462d-a9e6-99f1298b08c0-node-exporter-tls\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:35.978279 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:35.978258    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z5ql\" (UniqueName: \"kubernetes.io/projected/af725a92-e993-462d-a9e6-99f1298b08c0-kube-api-access-6z5ql\") pod \"node-exporter-nxhrj\" (UID: \"af725a92-e993-462d-a9e6-99f1298b08c0\") " pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:36.048346 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:36.048318    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:36.144991 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:36.144953    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nxhrj"
Jul 29 17:53:36.156573 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:53:36.156519    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf725a92_e993_462d_a9e6_99f1298b08c0.slice/crio-23c817a3a2c54cdd9104ee14cd842612ca3853de8b0874e08449d978dd8df05a WatchSource:0}: Error finding container 23c817a3a2c54cdd9104ee14cd842612ca3853de8b0874e08449d978dd8df05a: Status 404 returned error can't find the container with id 23c817a3a2c54cdd9104ee14cd842612ca3853de8b0874e08449d978dd8df05a
Jul 29 17:53:36.157864 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:36.157838    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:36.601503 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:36.601421    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:36.656643 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:36.656605    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxhrj" event={"ID":"af725a92-e993-462d-a9e6-99f1298b08c0","Type":"ContainerStarted","Data":"23c817a3a2c54cdd9104ee14cd842612ca3853de8b0874e08449d978dd8df05a"}
Jul 29 17:53:36.872703 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:36.872616    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:36.961580 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:36.961543    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:37.085134 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:37.085108    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:37.661565 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:37.661538    2556 generic.go:356] "Generic (PLEG): container finished" podID="af725a92-e993-462d-a9e6-99f1298b08c0" containerID="7bffec44eded68a2680c18189d45cbea5abe6aae73d4c4ebd9b0a652aeed4751" exitCode=0
Jul 29 17:53:37.661706 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:37.661627    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxhrj" event={"ID":"af725a92-e993-462d-a9e6-99f1298b08c0","Type":"ContainerDied","Data":"7bffec44eded68a2680c18189d45cbea5abe6aae73d4c4ebd9b0a652aeed4751"}
Jul 29 17:53:38.666610 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:38.666580    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxhrj" event={"ID":"af725a92-e993-462d-a9e6-99f1298b08c0","Type":"ContainerStarted","Data":"6dbc7f40a510e59a435d2755f0086dc3d0246f827fe87239040cb418fd4abb98"}
Jul 29 17:53:38.666997 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:38.666638    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxhrj" event={"ID":"af725a92-e993-462d-a9e6-99f1298b08c0","Type":"ContainerStarted","Data":"419c15dda1504837fbf3309b65c90f21b6ba38f18209f2da0cade48cd2a66b89"}
Jul 29 17:53:38.688658 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:38.688607    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nxhrj" podStartSLOduration=2.741984868 podStartE2EDuration="3.688582139s" totalImagesPullingTime="883.48534ms" totalInitContainerRuntime="63.111931ms" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:38.686675615 +0000 UTC m=+88.195059203" watchObservedRunningTime="2026-07-29 17:53:38.688582139 +0000 UTC m=+88.196965726"
Jul 29 17:53:38.797663 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:38.797632    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:38.902320 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:38.902288    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:39.005633 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:39.005550    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:39.104570 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:39.104540    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:39.375292 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:39.375209    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:39.451702 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:39.451671    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:39.519539 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:39.519513    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:39.638330 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:39.638301    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:39.693908 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:39.693868    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:39.837582 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:39.837547    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:39.932297 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:39.932211    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:39.987503 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:39.987478    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:40.248789 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:40.248705    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:40.300654 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:40.300610    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:40.468297 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:40.468269    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:40.559492 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:40.559403    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:40.680744 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:40.680715    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:40.779748 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:40.779699    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:41.038421 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:41.038387    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:41.095043 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:41.095016    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:41.248446 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:41.248417    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:41.451958 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:41.451931    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:41.767887 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:41.767801    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:41.829247 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:41.829213    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l" podUID="2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" containerName="registry" containerID="cri-o://db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702" gracePeriod=30
Jul 29 17:53:42.060486 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.060464    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:53:42.115620 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.115589    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-installation-pull-secrets\") pod \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") "
Jul 29 17:53:42.115620 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.115616    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls\") pod \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") "
Jul 29 17:53:42.115821 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.115638    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-image-registry-private-configuration\") pod \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") "
Jul 29 17:53:42.115821 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.115664    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-certificates\") pod \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") "
Jul 29 17:53:42.115924 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.115839    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv5cv\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-kube-api-access-xv5cv\") pod \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") "
Jul 29 17:53:42.115924 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.115887    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-ca-trust-extracted\") pod \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") "
Jul 29 17:53:42.115924 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.115911    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-trusted-ca\") pod \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") "
Jul 29 17:53:42.116064 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.115959    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-bound-sa-token\") pod \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\" (UID: \"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf\") "
Jul 29 17:53:42.116687 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.116429    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:53:42.118178 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.118131    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:53:42.119110 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.118412    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:53:42.119110 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.118560    2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-tls\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:53:42.119110 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.118583    2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-registry-certificates\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:53:42.119110 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.118599    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-trusted-ca\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:53:42.121417 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.120682    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:53:42.121417 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.120881    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:53:42.121417 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.120979    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-kube-api-access-xv5cv" (OuterVolumeSpecName: "kube-api-access-xv5cv") pod "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf"). InnerVolumeSpecName "kube-api-access-xv5cv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:53:42.124307 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.124279    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:53:42.125085 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.125063    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:42.129635 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.129605    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" (UID: "2a4db46e-c7d3-4b21-935e-b2e79d07c5cf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:53:42.219191 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.219144    2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-bound-sa-token\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:53:42.219191 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.219184    2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-installation-pull-secrets\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:53:42.219191 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.219198    2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-image-registry-private-configuration\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:53:42.219433 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.219210    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv5cv\" (UniqueName: \"kubernetes.io/projected/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-kube-api-access-xv5cv\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:53:42.219433 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.219219    2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf-ca-trust-extracted\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:53:42.680234 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.680201    2556 generic.go:356] "Generic (PLEG): container finished" podID="2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" containerID="db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702" exitCode=0
Jul 29 17:53:42.680410 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.680255    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l" event={"ID":"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf","Type":"ContainerDied","Data":"db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702"}
Jul 29 17:53:42.680410 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.680263    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l"
Jul 29 17:53:42.680410 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.680320    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f78b6ccd-vj58l" event={"ID":"2a4db46e-c7d3-4b21-935e-b2e79d07c5cf","Type":"ContainerDied","Data":"5ed0f7c528bb13e61aeaa1d83629e64f993f35395c6ea0dc3ac0fe19ad734102"}
Jul 29 17:53:42.680410 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.680341    2556 scope.go:117] "RemoveContainer" containerID="db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702"
Jul 29 17:53:42.688213 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.688197    2556 scope.go:117] "RemoveContainer" containerID="db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702"
Jul 29 17:53:42.688560 ip-10-0-135-253 kubenswrapper[2556]: E0729 17:53:42.688533    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702\": container with ID starting with db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702 not found: ID does not exist" containerID="db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702"
Jul 29 17:53:42.688654 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.688570    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702"} err="failed to get container status \"db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702\": rpc error: code = NotFound desc = could not find container \"db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702\": container with ID starting with db7e5a458f1985f77144defe163bf0869c6ca8f6add6b7c526a7a8f8a84d5702 not found: ID does not exist"
Jul 29 17:53:42.701131 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.701108    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67f78b6ccd-vj58l"]
Jul 29 17:53:42.706524 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:42.706504    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67f78b6ccd-vj58l"]
Jul 29 17:53:43.077737 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:43.077668    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" path="/var/lib/kubelet/pods/2a4db46e-c7d3-4b21-935e-b2e79d07c5cf/volumes"
Jul 29 17:53:43.326680 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:43.326649    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:43.393000 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:43.392973    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:44.617794 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:44.617762    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84f857b7cd-rfs4d"
Jul 29 17:53:44.776520 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:44.776493    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:44.894275 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:44.894236    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:46.135714 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:46.135682    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:46.220399 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:46.220353    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:48.737309 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:48.737278    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:48.838629 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:48.838600    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:51.758697 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:51.758668    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:51.824799 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:51.824770    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:53.610346 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:53.610317    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pv5rx"
Jul 29 17:53:57.469582 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:57.469545    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:53:57.590687 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:53:57.590657    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:02.279111 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:02.279076    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:02.379376 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:02.379332    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:06.681278 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:06.681245    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:06.784881 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:06.784853    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:06.861376 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:06.861323    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:08.533047 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:08.533018    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:17.152010 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:17.151976    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:17.215949 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:17.215907    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:17.891151 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:17.891079    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:17.992713 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:17.992669    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:21.751511 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:21.751475    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:21.846237 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:21.846205    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:55.115393 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:55.115348    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:54:55.429201 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:54:55.429169    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:04.019202 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:04.019122    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:04.077013 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:04.076982    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:04.268126 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:04.268089    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:04.369261 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:04.369179    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:04.752664 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:04.752627    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:04.807518 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:04.807493    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:14.693057 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:14.693021    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:14.794627 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:14.794595    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:14.850481 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:14.850455    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:36.351978 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:36.351945    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:36.409265 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:36.409233    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:55:44.503671 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:55:44.503631    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:10.438374 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:10.438328    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:10.519965 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:10.519936    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:13.754376 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:13.754334    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:13.854513 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:13.854481    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:13.954407 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:13.954352    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:14.448000 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:14.447970    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:14.554514 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:14.554487    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:22.326814 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:22.326778    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:22.391689 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:22.391655    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:24.959842 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:24.959806    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:25.062056 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:25.062021    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:25.161858 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:25.161830    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:36.786039 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:36.785954    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:56:36.883826 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:56:36.883786    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:10.915528 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:10.915502    2556 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 17:57:14.438736 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:14.438698    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:15.684873 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:15.684832    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:41.139891 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:41.139861    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:41.981736 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:41.981701    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:44.751219 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:44.751183    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:44.851119 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:44.851054    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:46.515976 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:46.515944    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:46.628818 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:46.628789    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:48.872305 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:48.872266    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:49.085610 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:49.085581    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:51.736430 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:51.736401    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:51.861427 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:51.861398    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:56.484976 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:56.484943    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:58.640595 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:58.640559    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:57:58.744573 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:57:58.744542    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:02.552052 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:02.552019    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:02.650166 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:02.650106    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:03.964621 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:03.964547    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c"]
Jul 29 17:58:03.965078 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:03.964779    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" containerName="registry"
Jul 29 17:58:03.965078 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:03.964789    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" containerName="registry"
Jul 29 17:58:03.965078 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:03.964835    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a4db46e-c7d3-4b21-935e-b2e79d07c5cf" containerName="registry"
Jul 29 17:58:03.967498 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:03.967484    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c"
Jul 29 17:58:03.970926 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:03.970900    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:58:03.971061 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:03.970903    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-dvj8j\""
Jul 29 17:58:03.971061 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:03.970911    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 17:58:03.976178 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:03.976160    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c"]
Jul 29 17:58:04.058141 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:04.058108    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svb9g\" (UniqueName: \"kubernetes.io/projected/acf028f5-4881-447d-b887-59e5e1b4d3d7-kube-api-access-svb9g\") pod \"openshift-lws-operator-fd8ccff4c-qrw2c\" (UID: \"acf028f5-4881-447d-b887-59e5e1b4d3d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c"
Jul 29 17:58:04.058318 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:04.058159    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acf028f5-4881-447d-b887-59e5e1b4d3d7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qrw2c\" (UID: \"acf028f5-4881-447d-b887-59e5e1b4d3d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c"
Jul 29 17:58:04.158501 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:04.158466    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acf028f5-4881-447d-b887-59e5e1b4d3d7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qrw2c\" (UID: \"acf028f5-4881-447d-b887-59e5e1b4d3d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c"
Jul 29 17:58:04.158640 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:04.158528    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svb9g\" (UniqueName: \"kubernetes.io/projected/acf028f5-4881-447d-b887-59e5e1b4d3d7-kube-api-access-svb9g\") pod \"openshift-lws-operator-fd8ccff4c-qrw2c\" (UID: \"acf028f5-4881-447d-b887-59e5e1b4d3d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c"
Jul 29 17:58:04.158874 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:04.158856    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acf028f5-4881-447d-b887-59e5e1b4d3d7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qrw2c\" (UID: \"acf028f5-4881-447d-b887-59e5e1b4d3d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c"
Jul 29 17:58:04.166837 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:04.166810    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svb9g\" (UniqueName: \"kubernetes.io/projected/acf028f5-4881-447d-b887-59e5e1b4d3d7-kube-api-access-svb9g\") pod \"openshift-lws-operator-fd8ccff4c-qrw2c\" (UID: \"acf028f5-4881-447d-b887-59e5e1b4d3d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c"
Jul 29 17:58:04.276105 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:04.276025    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c"
Jul 29 17:58:04.398778 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:04.398747    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c"]
Jul 29 17:58:04.401717 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:58:04.401688    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacf028f5_4881_447d_b887_59e5e1b4d3d7.slice/crio-cf24b0b81c02c77ea58502703cc5ed08b6c1b4a1112595f45fafcdfe14154672 WatchSource:0}: Error finding container cf24b0b81c02c77ea58502703cc5ed08b6c1b4a1112595f45fafcdfe14154672: Status 404 returned error can't find the container with id cf24b0b81c02c77ea58502703cc5ed08b6c1b4a1112595f45fafcdfe14154672
Jul 29 17:58:04.403407 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:04.403388    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 17:58:04.513126 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:04.513093    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c" event={"ID":"acf028f5-4881-447d-b887-59e5e1b4d3d7","Type":"ContainerStarted","Data":"cf24b0b81c02c77ea58502703cc5ed08b6c1b4a1112595f45fafcdfe14154672"}
Jul 29 17:58:07.525529 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:07.525497    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c" event={"ID":"acf028f5-4881-447d-b887-59e5e1b4d3d7","Type":"ContainerStarted","Data":"f315bbff759a2ea49eb1f9adcd3473eb7554cfe7683fbe17b9401f23f453610b"}
Jul 29 17:58:07.543451 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:07.543350    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qrw2c" podStartSLOduration=2.116817247 podStartE2EDuration="4.543335824s" totalImagesPullingTime="2.426518577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:07.542108696 +0000 UTC m=+357.050492285" watchObservedRunningTime="2026-07-29 17:58:07.543335824 +0000 UTC m=+357.051719412"
Jul 29 17:58:08.233812 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:08.233778    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:08.285331 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:08.285300    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:10.381544 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:10.381509    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:26.410220 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:26.410173    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:26.518889 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:26.518856    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:27.051699 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.051666    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:27.398516 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.398491    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:27.502160 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.502131    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:27.555518 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.555483    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"]
Jul 29 17:58:27.558939 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.558919    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.562454 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.562438    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9q8v4\""
Jul 29 17:58:27.562798 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.562773    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 17:58:27.562898 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.562774    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 17:58:27.562959 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.562935    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 17:58:27.563811 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.563794    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 17:58:27.580246 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.580225    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"]
Jul 29 17:58:27.628752 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.628723    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c775513b-de7a-40fb-8d99-2dcdac5526e4-manifests\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.628752 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.628755    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c775513b-de7a-40fb-8d99-2dcdac5526e4-tmp\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.628933 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.628773    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6snz\" (UniqueName: \"kubernetes.io/projected/c775513b-de7a-40fb-8d99-2dcdac5526e4-kube-api-access-j6snz\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.628933 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.628794    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c775513b-de7a-40fb-8d99-2dcdac5526e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.628933 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.628855    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c775513b-de7a-40fb-8d99-2dcdac5526e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.661628 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.661547    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:27.729791 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.729759    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c775513b-de7a-40fb-8d99-2dcdac5526e4-manifests\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.729939 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.729798    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c775513b-de7a-40fb-8d99-2dcdac5526e4-tmp\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.729939 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.729825    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6snz\" (UniqueName: \"kubernetes.io/projected/c775513b-de7a-40fb-8d99-2dcdac5526e4-kube-api-access-j6snz\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.729939 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.729858    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c775513b-de7a-40fb-8d99-2dcdac5526e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.729939 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.729890    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c775513b-de7a-40fb-8d99-2dcdac5526e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.730250 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.730229    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c775513b-de7a-40fb-8d99-2dcdac5526e4-manifests\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.732108 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.732085    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c775513b-de7a-40fb-8d99-2dcdac5526e4-tmp\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.732226 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.732210    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c775513b-de7a-40fb-8d99-2dcdac5526e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.732435 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.732415    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c775513b-de7a-40fb-8d99-2dcdac5526e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.741086 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.741062    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6snz\" (UniqueName: \"kubernetes.io/projected/c775513b-de7a-40fb-8d99-2dcdac5526e4-kube-api-access-j6snz\") pod \"opendatahub-operator-controller-manager-d84c6c79c-h7psx\" (UID: \"c775513b-de7a-40fb-8d99-2dcdac5526e4\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.760690 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.760663    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:27.862940 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.862909    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:27.868904 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.868874    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:27.972709 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:27.972675    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:28.017556 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:28.017526    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"]
Jul 29 17:58:28.021328 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:58:28.021300    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc775513b_de7a_40fb_8d99_2dcdac5526e4.slice/crio-e9d6dd005ddff7664ef52b15396f2c5cf5dcb2417f72c2d1a83e6a211099061d WatchSource:0}: Error finding container e9d6dd005ddff7664ef52b15396f2c5cf5dcb2417f72c2d1a83e6a211099061d: Status 404 returned error can't find the container with id e9d6dd005ddff7664ef52b15396f2c5cf5dcb2417f72c2d1a83e6a211099061d
Jul 29 17:58:28.085469 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:28.085443    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:28.591104 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:28.591069    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx" event={"ID":"c775513b-de7a-40fb-8d99-2dcdac5526e4","Type":"ContainerStarted","Data":"e9d6dd005ddff7664ef52b15396f2c5cf5dcb2417f72c2d1a83e6a211099061d"}
Jul 29 17:58:31.179752 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:31.179713    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:31.365168 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:31.365134    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:31.603471 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:31.603392    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx" event={"ID":"c775513b-de7a-40fb-8d99-2dcdac5526e4","Type":"ContainerStarted","Data":"07882ac42ffb3c72eabfe7594ae50d7251962c62cd2e92231751396d711f829a"}
Jul 29 17:58:32.608821 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:32.608735    2556 generic.go:356] "Generic (PLEG): container finished" podID="c775513b-de7a-40fb-8d99-2dcdac5526e4" containerID="07882ac42ffb3c72eabfe7594ae50d7251962c62cd2e92231751396d711f829a" exitCode=0
Jul 29 17:58:32.608821 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:32.608787    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx" event={"ID":"c775513b-de7a-40fb-8d99-2dcdac5526e4","Type":"ContainerDied","Data":"07882ac42ffb3c72eabfe7594ae50d7251962c62cd2e92231751396d711f829a"}
Jul 29 17:58:33.614208 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:33.614173    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx" event={"ID":"c775513b-de7a-40fb-8d99-2dcdac5526e4","Type":"ContainerStarted","Data":"aeeaf6c0fc38c939daf73976a9fe271dba3b7f9f63a0376290062124a249da49"}
Jul 29 17:58:33.614647 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:33.614320    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:33.636288 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:33.636229    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx" podStartSLOduration=2.017126613 podStartE2EDuration="6.636211962s" totalImagesPullingTime="3.6076373s" totalInitContainerRuntime="1.011448049s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:33.63473007 +0000 UTC m=+383.143113657" watchObservedRunningTime="2026-07-29 17:58:33.636211962 +0000 UTC m=+383.144595551"
Jul 29 17:58:38.985703 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:38.985666    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:39.080089 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:39.080059    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:43.358280 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:43.358249    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:43.418752 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:43.418724    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:44.619463 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:44.619436    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-h7psx"
Jul 29 17:58:46.982915 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:46.982866    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:47.274752 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.274680    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:47.578808 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.578723    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:47.685522 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.685490    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"]
Jul 29 17:58:47.693921 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.693900    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"
Jul 29 17:58:47.696518 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.696492    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 17:58:47.696663 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.696648    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 17:58:47.696857 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.696844    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 17:58:47.697728 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.697711    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 17:58:47.697783 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.697717    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-649kz\""
Jul 29 17:58:47.705091 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.705065    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"]
Jul 29 17:58:47.784725 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.784694    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4qw4\" (UniqueName: \"kubernetes.io/projected/d3825976-667d-40ba-95ec-48b5b62ece45-kube-api-access-b4qw4\") pod \"kube-auth-proxy-869f956dcc-7hq9m\" (UID: \"d3825976-667d-40ba-95ec-48b5b62ece45\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"
Jul 29 17:58:47.784886 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.784735    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3825976-667d-40ba-95ec-48b5b62ece45-tmp\") pod \"kube-auth-proxy-869f956dcc-7hq9m\" (UID: \"d3825976-667d-40ba-95ec-48b5b62ece45\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"
Jul 29 17:58:47.784886 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.784811    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3825976-667d-40ba-95ec-48b5b62ece45-tls-certs\") pod \"kube-auth-proxy-869f956dcc-7hq9m\" (UID: \"d3825976-667d-40ba-95ec-48b5b62ece45\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"
Jul 29 17:58:47.811170 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.811141    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:47.885874 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.885842    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3825976-667d-40ba-95ec-48b5b62ece45-tls-certs\") pod \"kube-auth-proxy-869f956dcc-7hq9m\" (UID: \"d3825976-667d-40ba-95ec-48b5b62ece45\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"
Jul 29 17:58:47.886044 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.885901    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4qw4\" (UniqueName: \"kubernetes.io/projected/d3825976-667d-40ba-95ec-48b5b62ece45-kube-api-access-b4qw4\") pod \"kube-auth-proxy-869f956dcc-7hq9m\" (UID: \"d3825976-667d-40ba-95ec-48b5b62ece45\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"
Jul 29 17:58:47.886044 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.885937    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3825976-667d-40ba-95ec-48b5b62ece45-tmp\") pod \"kube-auth-proxy-869f956dcc-7hq9m\" (UID: \"d3825976-667d-40ba-95ec-48b5b62ece45\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"
Jul 29 17:58:47.888221 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.888194    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3825976-667d-40ba-95ec-48b5b62ece45-tmp\") pod \"kube-auth-proxy-869f956dcc-7hq9m\" (UID: \"d3825976-667d-40ba-95ec-48b5b62ece45\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"
Jul 29 17:58:47.888355 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.888345    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3825976-667d-40ba-95ec-48b5b62ece45-tls-certs\") pod \"kube-auth-proxy-869f956dcc-7hq9m\" (UID: \"d3825976-667d-40ba-95ec-48b5b62ece45\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"
Jul 29 17:58:47.909931 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:47.909894    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4qw4\" (UniqueName: \"kubernetes.io/projected/d3825976-667d-40ba-95ec-48b5b62ece45-kube-api-access-b4qw4\") pod \"kube-auth-proxy-869f956dcc-7hq9m\" (UID: \"d3825976-667d-40ba-95ec-48b5b62ece45\") " pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"
Jul 29 17:58:48.003015 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:48.002979    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"
Jul 29 17:58:48.081589 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:48.081515    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:48.157107 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:48.157074    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m"]
Jul 29 17:58:48.160271 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:58:48.160233    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3825976_667d_40ba_95ec_48b5b62ece45.slice/crio-2e36e8d23a0548af6cfbdf48ad6248ba85deacf89808913c34cbe369c590a92b WatchSource:0}: Error finding container 2e36e8d23a0548af6cfbdf48ad6248ba85deacf89808913c34cbe369c590a92b: Status 404 returned error can't find the container with id 2e36e8d23a0548af6cfbdf48ad6248ba85deacf89808913c34cbe369c590a92b
Jul 29 17:58:48.184822 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:48.184796    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:48.285266 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:48.285233    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:48.432168 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:48.432074    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:48.668729 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:48.668692    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m" event={"ID":"d3825976-667d-40ba-95ec-48b5b62ece45","Type":"ContainerStarted","Data":"2e36e8d23a0548af6cfbdf48ad6248ba85deacf89808913c34cbe369c590a92b"}
Jul 29 17:58:48.944514 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:48.944474    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:49.087889 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.087854    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"]
Jul 29 17:58:49.091314 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.091291    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.093562 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.093538    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:49.093876 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.093861    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 29 17:58:49.094180 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.094019    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lsscf\""
Jul 29 17:58:49.094180 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.094098    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 29 17:58:49.094787 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.094766    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 17:58:49.094892 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.094793    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 29 17:58:49.103929 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.103905    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"]
Jul 29 17:58:49.164923 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.164888    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:49.198200 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.198112    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.198200 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.198177    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.198442 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.198281    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/210ea334-1b3c-4384-9625-9bcf03028b88-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.198442 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.198350    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.198442 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.198407    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.198587 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.198451    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.198587 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.198486    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs64k\" (UniqueName: \"kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-kube-api-access-bs64k\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.299705 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.299670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.299893 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.299718    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/210ea334-1b3c-4384-9625-9bcf03028b88-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.299893 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.299746    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.299893 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.299763    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.299893 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.299786    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.299893 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.299808    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs64k\" (UniqueName: \"kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-kube-api-access-bs64k\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.299893 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.299834    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.300488 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.300457    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.302280 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.302255    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.302403 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.302339    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.302500 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.302481    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/210ea334-1b3c-4384-9625-9bcf03028b88-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.302660 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.302642    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.308764 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.308742    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.309271 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.309237    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs64k\" (UniqueName: \"kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-kube-api-access-bs64k\") pod \"istiod-openshift-gateway-7b7995d5cd-f5tsk\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.404090 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.404053    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:49.483292 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.483254    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:49.575059 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.575025    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"]
Jul 29 17:58:49.578427 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:58:49.578395    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210ea334_1b3c_4384_9625_9bcf03028b88.slice/crio-938ddf7aa98d84830075d225f767bb35ab75bf7e920b9145a895920033d4e43e WatchSource:0}: Error finding container 938ddf7aa98d84830075d225f767bb35ab75bf7e920b9145a895920033d4e43e: Status 404 returned error can't find the container with id 938ddf7aa98d84830075d225f767bb35ab75bf7e920b9145a895920033d4e43e
Jul 29 17:58:49.675382 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:49.675331    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk" event={"ID":"210ea334-1b3c-4384-9625-9bcf03028b88","Type":"ContainerStarted","Data":"938ddf7aa98d84830075d225f767bb35ab75bf7e920b9145a895920033d4e43e"}
Jul 29 17:58:50.087377 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:50.087337    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:51.685919 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:51.685877    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m" event={"ID":"d3825976-667d-40ba-95ec-48b5b62ece45","Type":"ContainerStarted","Data":"b0c1bd4b2f80c8454eafb4df6b3f5e77ebfaed0a97e3d0eb06b39e7b4593eb5d"}
Jul 29 17:58:51.702120 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:51.702069    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-869f956dcc-7hq9m" podStartSLOduration=2.21878364 podStartE2EDuration="4.702047663s" totalImagesPullingTime="2.483264023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:51.70099971 +0000 UTC m=+401.209383316" watchObservedRunningTime="2026-07-29 17:58:51.702047663 +0000 UTC m=+401.210431254"
Jul 29 17:58:52.467571 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:52.467533    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 29 17:58:52.467670 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:52.467600    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 29 17:58:52.691712 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:52.691676    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk" event={"ID":"210ea334-1b3c-4384-9625-9bcf03028b88","Type":"ContainerStarted","Data":"72756a8b884b5939634bf495ec003ab2460faa2deea7b94d28e78b3c632c6256"}
Jul 29 17:58:52.692108 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:52.691804    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:52.717473 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:52.717379    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk" podStartSLOduration=0.83074583 podStartE2EDuration="3.717352298s" totalImagesPullingTime="2.886606468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:52.715182117 +0000 UTC m=+402.223565706" watchObservedRunningTime="2026-07-29 17:58:52.717352298 +0000 UTC m=+402.225735885"
Jul 29 17:58:53.295382 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:53.295328    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:53.478766 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:53.478733    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:53.696602 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:53.696565    2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-f5tsk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 17:58:53.696963 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:53.696633    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk" podUID="210ea334-1b3c-4384-9625-9bcf03028b88" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 17:58:54.067309 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:54.067200    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:54.324669 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:54.324545    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:54.428205 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:54.428175    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:56.233963 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.233930    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"]
Jul 29 17:58:56.237246 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.237230    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.243419 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.243389    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 17:58:56.243796 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.243772    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 17:58:56.243900 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.243874    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 17:58:56.243970 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.243916    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ddj8l\""
Jul 29 17:58:56.263542 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.263516    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"]
Jul 29 17:58:56.368353 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.368325    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-metrics-cert\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.368534 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.368357    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-cert\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.368534 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.368459    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfgn2\" (UniqueName: \"kubernetes.io/projected/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-kube-api-access-xfgn2\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.368534 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.368487    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-manager-config\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.469419 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.469386    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-metrics-cert\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.469594 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.469427    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-cert\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.469594 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.469495    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfgn2\" (UniqueName: \"kubernetes.io/projected/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-kube-api-access-xfgn2\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.469594 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.469525    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-manager-config\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.470265 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.470238    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-manager-config\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.471922 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.471899    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-cert\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.472029 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.471975    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-metrics-cert\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.478001 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.477976    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfgn2\" (UniqueName: \"kubernetes.io/projected/27c763e9-bdd5-4f9d-b384-a1c359cad1e8-kube-api-access-xfgn2\") pod \"lws-controller-manager-7ff9fc5cb5-2rzm8\" (UID: \"27c763e9-bdd5-4f9d-b384-a1c359cad1e8\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.549708 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.549603    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:56.689096 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.689069    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"]
Jul 29 17:58:56.691533 ip-10-0-135-253 kubenswrapper[2556]: W0729 17:58:56.691506    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c763e9_bdd5_4f9d_b384_a1c359cad1e8.slice/crio-c4e72dd0c6707e62168297c95526ac7be0cbcdeab65211778664164639025265 WatchSource:0}: Error finding container c4e72dd0c6707e62168297c95526ac7be0cbcdeab65211778664164639025265: Status 404 returned error can't find the container with id c4e72dd0c6707e62168297c95526ac7be0cbcdeab65211778664164639025265
Jul 29 17:58:56.695614 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.695591    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:58:56.706869 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:56.706847    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8" event={"ID":"27c763e9-bdd5-4f9d-b384-a1c359cad1e8","Type":"ContainerStarted","Data":"c4e72dd0c6707e62168297c95526ac7be0cbcdeab65211778664164639025265"}
Jul 29 17:58:58.707881 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:58.707841    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:58.730087 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:58.729755    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8" event={"ID":"27c763e9-bdd5-4f9d-b384-a1c359cad1e8","Type":"ContainerStarted","Data":"fda3280a8e4b6b0b579f01e924af2ae9596546d4b0dcf4e29251de088fb062d7"}
Jul 29 17:58:58.730087 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:58.729868    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:58:58.826788 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:58.826760    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:58.984000 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:58.983967    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:58:59.249684 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:58:59.249600    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:59:00.102137 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:00.102107    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:59:00.216228 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:00.216195    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:59:00.274424 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:00.274389    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:59:01.615525 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:01.615467    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8" podStartSLOduration=3.707842585 podStartE2EDuration="5.61544574s" totalImagesPullingTime="1.907603155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:58.772571173 +0000 UTC m=+408.280954762" watchObservedRunningTime="2026-07-29 17:59:01.61544574 +0000 UTC m=+411.123829329"
Jul 29 17:59:01.617324 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:01.617298    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"]
Jul 29 17:59:01.617494 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:01.617474    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk" podUID="210ea334-1b3c-4384-9625-9bcf03028b88" containerName="discovery" containerID="cri-o://72756a8b884b5939634bf495ec003ab2460faa2deea7b94d28e78b3c632c6256" gracePeriod=30
Jul 29 17:59:01.742012 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:01.741985    2556 generic.go:356] "Generic (PLEG): container finished" podID="210ea334-1b3c-4384-9625-9bcf03028b88" containerID="72756a8b884b5939634bf495ec003ab2460faa2deea7b94d28e78b3c632c6256" exitCode=0
Jul 29 17:59:01.742133 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:01.742059    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk" event={"ID":"210ea334-1b3c-4384-9625-9bcf03028b88","Type":"ContainerDied","Data":"72756a8b884b5939634bf495ec003ab2460faa2deea7b94d28e78b3c632c6256"}
Jul 29 17:59:01.870824 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:01.870770    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:59:02.018041 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.018007    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-kubeconfig\") pod \"210ea334-1b3c-4384-9625-9bcf03028b88\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") "
Jul 29 17:59:02.018234 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.018062    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/210ea334-1b3c-4384-9625-9bcf03028b88-local-certs\") pod \"210ea334-1b3c-4384-9625-9bcf03028b88\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") "
Jul 29 17:59:02.018234 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.018082    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-istio-token\") pod \"210ea334-1b3c-4384-9625-9bcf03028b88\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") "
Jul 29 17:59:02.018234 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.018102    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-cacerts\") pod \"210ea334-1b3c-4384-9625-9bcf03028b88\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") "
Jul 29 17:59:02.018234 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.018130    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs64k\" (UniqueName: \"kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-kube-api-access-bs64k\") pod \"210ea334-1b3c-4384-9625-9bcf03028b88\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") "
Jul 29 17:59:02.018474 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.018326    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-dns-cert\") pod \"210ea334-1b3c-4384-9625-9bcf03028b88\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") "
Jul 29 17:59:02.018474 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.018412    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-ca-configmap\") pod \"210ea334-1b3c-4384-9625-9bcf03028b88\" (UID: \"210ea334-1b3c-4384-9625-9bcf03028b88\") "
Jul 29 17:59:02.018869 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.018831    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "210ea334-1b3c-4384-9625-9bcf03028b88" (UID: "210ea334-1b3c-4384-9625-9bcf03028b88"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:59:02.020529 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.020500    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/210ea334-1b3c-4384-9625-9bcf03028b88-local-certs" (OuterVolumeSpecName: "local-certs") pod "210ea334-1b3c-4384-9625-9bcf03028b88" (UID: "210ea334-1b3c-4384-9625-9bcf03028b88"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:59:02.020650 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.020579    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "210ea334-1b3c-4384-9625-9bcf03028b88" (UID: "210ea334-1b3c-4384-9625-9bcf03028b88"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:59:02.020765 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.020731    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "210ea334-1b3c-4384-9625-9bcf03028b88" (UID: "210ea334-1b3c-4384-9625-9bcf03028b88"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:59:02.020855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.020806    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-istio-token" (OuterVolumeSpecName: "istio-token") pod "210ea334-1b3c-4384-9625-9bcf03028b88" (UID: "210ea334-1b3c-4384-9625-9bcf03028b88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:59:02.020855 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.020805    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-cacerts" (OuterVolumeSpecName: "cacerts") pod "210ea334-1b3c-4384-9625-9bcf03028b88" (UID: "210ea334-1b3c-4384-9625-9bcf03028b88"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:59:02.021012 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.020990    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-kube-api-access-bs64k" (OuterVolumeSpecName: "kube-api-access-bs64k") pod "210ea334-1b3c-4384-9625-9bcf03028b88" (UID: "210ea334-1b3c-4384-9625-9bcf03028b88"). InnerVolumeSpecName "kube-api-access-bs64k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:59:02.119640 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.119597    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs64k\" (UniqueName: \"kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-kube-api-access-bs64k\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:59:02.119640 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.119634    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-dns-cert\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:59:02.119640 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.119643    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/210ea334-1b3c-4384-9625-9bcf03028b88-istio-csr-ca-configmap\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:59:02.119640 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.119652    2556 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-istio-kubeconfig\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:59:02.119895 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.119661    2556 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/210ea334-1b3c-4384-9625-9bcf03028b88-local-certs\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:59:02.119895 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.119670    2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/210ea334-1b3c-4384-9625-9bcf03028b88-istio-token\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:59:02.119895 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.119677    2556 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/210ea334-1b3c-4384-9625-9bcf03028b88-cacerts\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 17:59:02.746977 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.746949    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"
Jul 29 17:59:02.746977 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.746966    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk" event={"ID":"210ea334-1b3c-4384-9625-9bcf03028b88","Type":"ContainerDied","Data":"938ddf7aa98d84830075d225f767bb35ab75bf7e920b9145a895920033d4e43e"}
Jul 29 17:59:02.747493 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.747018    2556 scope.go:117] "RemoveContainer" containerID="72756a8b884b5939634bf495ec003ab2460faa2deea7b94d28e78b3c632c6256"
Jul 29 17:59:02.794883 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.791902    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"]
Jul 29 17:59:02.799517 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:02.799494    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f5tsk"]
Jul 29 17:59:03.078014 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:03.077930    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="210ea334-1b3c-4384-9625-9bcf03028b88" path="/var/lib/kubelet/pods/210ea334-1b3c-4384-9625-9bcf03028b88/volumes"
Jul 29 17:59:09.736338 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:09.736309    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-2rzm8"
Jul 29 17:59:21.931836 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:21.931807    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:59:22.004666 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:22.004636    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:59:48.827856 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:48.827760    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:59:48.954458 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:48.954430    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 17:59:49.070590 ip-10-0-135-253 kubenswrapper[2556]: I0729 17:59:49.070563    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:09.528246 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:09.528212    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:09.629142 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:09.629110    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:13.165349 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:13.165290    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:19.289864 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:19.289826    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:20.132209 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:20.132176    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:20.233448 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:20.233412    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:20.334247 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:20.334215    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:22.893617 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:22.893582    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:23.704260 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:23.704223    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:25.161205 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:25.161172    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:27.487280 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:27.487241    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:27.902815 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:27.902787    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:29.345647 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:29.345611    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:31.117547 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:31.117512    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:31.722859 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:31.722825    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:33.863444 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:33.863408    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:35.091855 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:35.091820    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:35.192723 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:35.192692    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:36.009929 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.009895    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"]
Jul 29 18:00:36.010165 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.010153    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="210ea334-1b3c-4384-9625-9bcf03028b88" containerName="discovery"
Jul 29 18:00:36.010212 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.010167    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="210ea334-1b3c-4384-9625-9bcf03028b88" containerName="discovery"
Jul 29 18:00:36.010245 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.010223    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="210ea334-1b3c-4384-9625-9bcf03028b88" containerName="discovery"
Jul 29 18:00:36.012914 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.012898    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:36.015562 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.015534    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 29 18:00:36.015712 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.015689    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 18:00:36.015770 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.015731    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 18:00:36.016881 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.016860    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 29 18:00:36.016991 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.016922    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dz6bj\""
Jul 29 18:00:36.028229 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.028209    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"]
Jul 29 18:00:36.052667 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.052639    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:36.128689 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.128658    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0533fe48-2d39-42f5-af54-ad93e6de2f8a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mf6sk\" (UID: \"0533fe48-2d39-42f5-af54-ad93e6de2f8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:36.128689 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.128695    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0533fe48-2d39-42f5-af54-ad93e6de2f8a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mf6sk\" (UID: \"0533fe48-2d39-42f5-af54-ad93e6de2f8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:36.129122 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.128740    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6mnf\" (UniqueName: \"kubernetes.io/projected/0533fe48-2d39-42f5-af54-ad93e6de2f8a-kube-api-access-k6mnf\") pod \"kuadrant-console-plugin-66b5774985-mf6sk\" (UID: \"0533fe48-2d39-42f5-af54-ad93e6de2f8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:36.229886 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.229857    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mnf\" (UniqueName: \"kubernetes.io/projected/0533fe48-2d39-42f5-af54-ad93e6de2f8a-kube-api-access-k6mnf\") pod \"kuadrant-console-plugin-66b5774985-mf6sk\" (UID: \"0533fe48-2d39-42f5-af54-ad93e6de2f8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:36.230059 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.229951    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0533fe48-2d39-42f5-af54-ad93e6de2f8a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mf6sk\" (UID: \"0533fe48-2d39-42f5-af54-ad93e6de2f8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:36.230059 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.229978    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0533fe48-2d39-42f5-af54-ad93e6de2f8a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mf6sk\" (UID: \"0533fe48-2d39-42f5-af54-ad93e6de2f8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:36.230182 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:00:36.230105    2556 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 29 18:00:36.230233 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:00:36.230209    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0533fe48-2d39-42f5-af54-ad93e6de2f8a-plugin-serving-cert podName:0533fe48-2d39-42f5-af54-ad93e6de2f8a nodeName:}" failed. No retries permitted until 2026-07-29 18:00:36.730187186 +0000 UTC m=+506.238570775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/0533fe48-2d39-42f5-af54-ad93e6de2f8a-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-mf6sk" (UID: "0533fe48-2d39-42f5-af54-ad93e6de2f8a") : secret "plugin-serving-cert" not found
Jul 29 18:00:36.230658 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.230627    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0533fe48-2d39-42f5-af54-ad93e6de2f8a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mf6sk\" (UID: \"0533fe48-2d39-42f5-af54-ad93e6de2f8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:36.238271 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.238249    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6mnf\" (UniqueName: \"kubernetes.io/projected/0533fe48-2d39-42f5-af54-ad93e6de2f8a-kube-api-access-k6mnf\") pod \"kuadrant-console-plugin-66b5774985-mf6sk\" (UID: \"0533fe48-2d39-42f5-af54-ad93e6de2f8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:36.607480 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.607452    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:36.734196 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.734163    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0533fe48-2d39-42f5-af54-ad93e6de2f8a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mf6sk\" (UID: \"0533fe48-2d39-42f5-af54-ad93e6de2f8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:36.739348 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.737551    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0533fe48-2d39-42f5-af54-ad93e6de2f8a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mf6sk\" (UID: \"0533fe48-2d39-42f5-af54-ad93e6de2f8a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:36.921670 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:36.921635    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"
Jul 29 18:00:37.045023 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:37.044987    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk"]
Jul 29 18:00:37.047878 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:00:37.047846    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0533fe48_2d39_42f5_af54_ad93e6de2f8a.slice/crio-b562da6c48cb55fe12c8ea6db289cc88f7688f333b055774264cb44cc90975b3 WatchSource:0}: Error finding container b562da6c48cb55fe12c8ea6db289cc88f7688f333b055774264cb44cc90975b3: Status 404 returned error can't find the container with id b562da6c48cb55fe12c8ea6db289cc88f7688f333b055774264cb44cc90975b3
Jul 29 18:00:37.083807 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:37.083777    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk" event={"ID":"0533fe48-2d39-42f5-af54-ad93e6de2f8a","Type":"ContainerStarted","Data":"b562da6c48cb55fe12c8ea6db289cc88f7688f333b055774264cb44cc90975b3"}
Jul 29 18:00:38.505781 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:38.505668    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:38.569794 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:38.569760    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:39.399037 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:39.399000    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:40.177866 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:40.177827    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:47.186911 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:47.186825    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:47.285893 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:47.285828    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:49.589967 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:49.589913    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:49.689405 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:49.689343    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:53.940293 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:53.940249    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:54.044304 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:54.044249    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:56.169266 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:56.169234    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:57.787987 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:57.787944    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:00:59.178707 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:59.178674    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk" event={"ID":"0533fe48-2d39-42f5-af54-ad93e6de2f8a","Type":"ContainerStarted","Data":"01a1421650c0a3b9a7643e0af4a6bc8590d549853655ceccbe7273451f4640f9"}
Jul 29 18:00:59.201888 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:00:59.201841    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mf6sk" podStartSLOduration=2.606115815 podStartE2EDuration="24.201826115s" totalImagesPullingTime="21.5957103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:00:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:00:59.200419448 +0000 UTC m=+528.708803035" watchObservedRunningTime="2026-07-29 18:00:59.201826115 +0000 UTC m=+528.710209703"
Jul 29 18:01:04.917796 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:04.917698    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:05.019154 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:05.019124    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:07.375438 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:07.375400    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:07.474993 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:07.474951    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:11.700672 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:11.700644    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:11.801003 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:11.800965    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:14.720133 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:14.720091    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:14.785783 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:14.785739    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:15.631383 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:15.631323    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:15.773841 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:15.773815    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:17.323798 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:17.323758    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:17.437393 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:17.437351    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:17.537472 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:17.537440    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:18.192705 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.192673    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-7qgrk"]
Jul 29 18:01:18.239423 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.239385    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qgrk"]
Jul 29 18:01:18.239571 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.239474    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qgrk"
Jul 29 18:01:18.242041 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.242021    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8j46j\""
Jul 29 18:01:18.328005 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.327973    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-nxzpp"]
Jul 29 18:01:18.331277 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.331258    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nxzpp"
Jul 29 18:01:18.344462 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.344439    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nxzpp"]
Jul 29 18:01:18.379063 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.379035    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9w7l\" (UniqueName: \"kubernetes.io/projected/d6196b1a-4186-4803-bcc2-cd64caa0c335-kube-api-access-n9w7l\") pod \"authorino-58cd955675-7qgrk\" (UID: \"d6196b1a-4186-4803-bcc2-cd64caa0c335\") " pod="kuadrant-system/authorino-58cd955675-7qgrk"
Jul 29 18:01:18.480195 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.480106    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd6cd\" (UniqueName: \"kubernetes.io/projected/969e560e-4a94-4a96-ad93-1249436bd1ba-kube-api-access-rd6cd\") pod \"authorino-75d556bd77-nxzpp\" (UID: \"969e560e-4a94-4a96-ad93-1249436bd1ba\") " pod="kuadrant-system/authorino-75d556bd77-nxzpp"
Jul 29 18:01:18.480195 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.480154    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9w7l\" (UniqueName: \"kubernetes.io/projected/d6196b1a-4186-4803-bcc2-cd64caa0c335-kube-api-access-n9w7l\") pod \"authorino-58cd955675-7qgrk\" (UID: \"d6196b1a-4186-4803-bcc2-cd64caa0c335\") " pod="kuadrant-system/authorino-58cd955675-7qgrk"
Jul 29 18:01:18.488343 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.488320    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9w7l\" (UniqueName: \"kubernetes.io/projected/d6196b1a-4186-4803-bcc2-cd64caa0c335-kube-api-access-n9w7l\") pod \"authorino-58cd955675-7qgrk\" (UID: \"d6196b1a-4186-4803-bcc2-cd64caa0c335\") " pod="kuadrant-system/authorino-58cd955675-7qgrk"
Jul 29 18:01:18.548873 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.548840    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qgrk"
Jul 29 18:01:18.581292 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.581245    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd6cd\" (UniqueName: \"kubernetes.io/projected/969e560e-4a94-4a96-ad93-1249436bd1ba-kube-api-access-rd6cd\") pod \"authorino-75d556bd77-nxzpp\" (UID: \"969e560e-4a94-4a96-ad93-1249436bd1ba\") " pod="kuadrant-system/authorino-75d556bd77-nxzpp"
Jul 29 18:01:18.590620 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.590592    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd6cd\" (UniqueName: \"kubernetes.io/projected/969e560e-4a94-4a96-ad93-1249436bd1ba-kube-api-access-rd6cd\") pod \"authorino-75d556bd77-nxzpp\" (UID: \"969e560e-4a94-4a96-ad93-1249436bd1ba\") " pod="kuadrant-system/authorino-75d556bd77-nxzpp"
Jul 29 18:01:18.639861 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.639831    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nxzpp"
Jul 29 18:01:18.681033 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.680967    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qgrk"]
Jul 29 18:01:18.683960 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:01:18.683932    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6196b1a_4186_4803_bcc2_cd64caa0c335.slice/crio-a82578179c91b71ccb1d7afb60f9bb18a3b7894044e38971e0ef7012cbf118ff WatchSource:0}: Error finding container a82578179c91b71ccb1d7afb60f9bb18a3b7894044e38971e0ef7012cbf118ff: Status 404 returned error can't find the container with id a82578179c91b71ccb1d7afb60f9bb18a3b7894044e38971e0ef7012cbf118ff
Jul 29 18:01:18.780466 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:18.780443    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nxzpp"]
Jul 29 18:01:18.782148 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:01:18.782116    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod969e560e_4a94_4a96_ad93_1249436bd1ba.slice/crio-0adc939408090b538d7a2f312387e6c6c0ce9c4f1a5cb1b3d8ffa7277207c9ab WatchSource:0}: Error finding container 0adc939408090b538d7a2f312387e6c6c0ce9c4f1a5cb1b3d8ffa7277207c9ab: Status 404 returned error can't find the container with id 0adc939408090b538d7a2f312387e6c6c0ce9c4f1a5cb1b3d8ffa7277207c9ab
Jul 29 18:01:19.248767 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:19.248734    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qgrk" event={"ID":"d6196b1a-4186-4803-bcc2-cd64caa0c335","Type":"ContainerStarted","Data":"a82578179c91b71ccb1d7afb60f9bb18a3b7894044e38971e0ef7012cbf118ff"}
Jul 29 18:01:19.249802 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:19.249780    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nxzpp" event={"ID":"969e560e-4a94-4a96-ad93-1249436bd1ba","Type":"ContainerStarted","Data":"0adc939408090b538d7a2f312387e6c6c0ce9c4f1a5cb1b3d8ffa7277207c9ab"}
Jul 29 18:01:19.439330 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:19.439292    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:19.542141 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:19.542015    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:23.269235 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:23.269197    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qgrk" event={"ID":"d6196b1a-4186-4803-bcc2-cd64caa0c335","Type":"ContainerStarted","Data":"87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269"}
Jul 29 18:01:23.270729 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:23.270701    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nxzpp" event={"ID":"969e560e-4a94-4a96-ad93-1249436bd1ba","Type":"ContainerStarted","Data":"01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959"}
Jul 29 18:01:23.286145 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:23.286097    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-7qgrk" podStartSLOduration=1.618718047 podStartE2EDuration="5.286081685s" totalImagesPullingTime="3.667363638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:23.284652932 +0000 UTC m=+552.793036519" watchObservedRunningTime="2026-07-29 18:01:23.286081685 +0000 UTC m=+552.794465273"
Jul 29 18:01:23.300718 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:23.300668    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-nxzpp" podStartSLOduration=1.721306947 podStartE2EDuration="5.300653073s" totalImagesPullingTime="3.579346126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:23.300200018 +0000 UTC m=+552.808583595" watchObservedRunningTime="2026-07-29 18:01:23.300653073 +0000 UTC m=+552.809036660"
Jul 29 18:01:23.335103 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:23.335072    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qgrk"]
Jul 29 18:01:23.962418 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:23.962385    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:24.064486 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:24.064449    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:25.279293 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:25.279228    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-7qgrk" podUID="d6196b1a-4186-4803-bcc2-cd64caa0c335" containerName="authorino" containerID="cri-o://87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269" gracePeriod=30
Jul 29 18:01:25.517702 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:25.517676    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qgrk"
Jul 29 18:01:25.638432 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:25.638395    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9w7l\" (UniqueName: \"kubernetes.io/projected/d6196b1a-4186-4803-bcc2-cd64caa0c335-kube-api-access-n9w7l\") pod \"d6196b1a-4186-4803-bcc2-cd64caa0c335\" (UID: \"d6196b1a-4186-4803-bcc2-cd64caa0c335\") "
Jul 29 18:01:25.640554 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:25.640532    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6196b1a-4186-4803-bcc2-cd64caa0c335-kube-api-access-n9w7l" (OuterVolumeSpecName: "kube-api-access-n9w7l") pod "d6196b1a-4186-4803-bcc2-cd64caa0c335" (UID: "d6196b1a-4186-4803-bcc2-cd64caa0c335"). InnerVolumeSpecName "kube-api-access-n9w7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:01:25.739610 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:25.739584    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9w7l\" (UniqueName: \"kubernetes.io/projected/d6196b1a-4186-4803-bcc2-cd64caa0c335-kube-api-access-n9w7l\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:01:26.284912 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:26.284876    2556 generic.go:356] "Generic (PLEG): container finished" podID="d6196b1a-4186-4803-bcc2-cd64caa0c335" containerID="87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269" exitCode=0
Jul 29 18:01:26.285335 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:26.284924    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qgrk"
Jul 29 18:01:26.285335 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:26.284923    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qgrk" event={"ID":"d6196b1a-4186-4803-bcc2-cd64caa0c335","Type":"ContainerDied","Data":"87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269"}
Jul 29 18:01:26.285335 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:26.285043    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qgrk" event={"ID":"d6196b1a-4186-4803-bcc2-cd64caa0c335","Type":"ContainerDied","Data":"a82578179c91b71ccb1d7afb60f9bb18a3b7894044e38971e0ef7012cbf118ff"}
Jul 29 18:01:26.285335 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:26.285071    2556 scope.go:117] "RemoveContainer" containerID="87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269"
Jul 29 18:01:26.293023 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:26.293005    2556 scope.go:117] "RemoveContainer" containerID="87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269"
Jul 29 18:01:26.293249 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:01:26.293230    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269\": container with ID starting with 87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269 not found: ID does not exist" containerID="87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269"
Jul 29 18:01:26.293306 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:26.293257    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269"} err="failed to get container status \"87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269\": rpc error: code = NotFound desc = could not find container \"87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269\": container with ID starting with 87f31d9e60ac8dbb7ff8c22e60f6a95b9f8fc33a1d31273d69f8db83f005c269 not found: ID does not exist"
Jul 29 18:01:26.315533 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:26.315505    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qgrk"]
Jul 29 18:01:26.325118 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:26.325097    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qgrk"]
Jul 29 18:01:26.390023 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:26.389963    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:26.491263 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:26.491232    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:27.078354 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:27.078326    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6196b1a-4186-4803-bcc2-cd64caa0c335" path="/var/lib/kubelet/pods/d6196b1a-4186-4803-bcc2-cd64caa0c335/volumes"
Jul 29 18:01:33.453720 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:33.453692    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:33.559738 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:33.559705    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:33.903128 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:33.903080    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:45.756702 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:45.756661    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:45.978210 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:45.978174    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-khrg7"]
Jul 29 18:01:45.978502 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:45.978487    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6196b1a-4186-4803-bcc2-cd64caa0c335" containerName="authorino"
Jul 29 18:01:45.978556 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:45.978504    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6196b1a-4186-4803-bcc2-cd64caa0c335" containerName="authorino"
Jul 29 18:01:45.978590 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:45.978571    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6196b1a-4186-4803-bcc2-cd64caa0c335" containerName="authorino"
Jul 29 18:01:45.980322 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:45.980305    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-khrg7"
Jul 29 18:01:45.985372 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:45.985323    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6kx7\" (UniqueName: \"kubernetes.io/projected/484aad4e-0cbf-4f26-9e12-df31b65c4126-kube-api-access-v6kx7\") pod \"authorino-5d4b67c446-khrg7\" (UID: \"484aad4e-0cbf-4f26-9e12-df31b65c4126\") " pod="kuadrant-system/authorino-5d4b67c446-khrg7"
Jul 29 18:01:45.989733 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:45.989709    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-khrg7"]
Jul 29 18:01:46.086199 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.086121    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6kx7\" (UniqueName: \"kubernetes.io/projected/484aad4e-0cbf-4f26-9e12-df31b65c4126-kube-api-access-v6kx7\") pod \"authorino-5d4b67c446-khrg7\" (UID: \"484aad4e-0cbf-4f26-9e12-df31b65c4126\") " pod="kuadrant-system/authorino-5d4b67c446-khrg7"
Jul 29 18:01:46.094285 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.094266    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6kx7\" (UniqueName: \"kubernetes.io/projected/484aad4e-0cbf-4f26-9e12-df31b65c4126-kube-api-access-v6kx7\") pod \"authorino-5d4b67c446-khrg7\" (UID: \"484aad4e-0cbf-4f26-9e12-df31b65c4126\") " pod="kuadrant-system/authorino-5d4b67c446-khrg7"
Jul 29 18:01:46.211142 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.211108    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-khrg7"]
Jul 29 18:01:46.211341 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.211329    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-khrg7"
Jul 29 18:01:46.239100 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.238601    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54d999fbc7-28znk"]
Jul 29 18:01:46.240962 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.240938    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54d999fbc7-28znk"
Jul 29 18:01:46.248506 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.248479    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54d999fbc7-28znk"]
Jul 29 18:01:46.288436 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.288404    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hb4p\" (UniqueName: \"kubernetes.io/projected/45d04d32-b6c3-49a5-b23e-496d6314a0fa-kube-api-access-4hb4p\") pod \"authorino-54d999fbc7-28znk\" (UID: \"45d04d32-b6c3-49a5-b23e-496d6314a0fa\") " pod="kuadrant-system/authorino-54d999fbc7-28znk"
Jul 29 18:01:46.301619 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.301597    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:46.350120 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.350049    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-khrg7"]
Jul 29 18:01:46.352273 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:01:46.352249    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod484aad4e_0cbf_4f26_9e12_df31b65c4126.slice/crio-07c4ef29e3e7f91bd1789c8745291b883c76ca74ce7b1b692b0416a46fff3e18 WatchSource:0}: Error finding container 07c4ef29e3e7f91bd1789c8745291b883c76ca74ce7b1b692b0416a46fff3e18: Status 404 returned error can't find the container with id 07c4ef29e3e7f91bd1789c8745291b883c76ca74ce7b1b692b0416a46fff3e18
Jul 29 18:01:46.362028 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.362001    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-khrg7" event={"ID":"484aad4e-0cbf-4f26-9e12-df31b65c4126","Type":"ContainerStarted","Data":"07c4ef29e3e7f91bd1789c8745291b883c76ca74ce7b1b692b0416a46fff3e18"}
Jul 29 18:01:46.389611 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.389585    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hb4p\" (UniqueName: \"kubernetes.io/projected/45d04d32-b6c3-49a5-b23e-496d6314a0fa-kube-api-access-4hb4p\") pod \"authorino-54d999fbc7-28znk\" (UID: \"45d04d32-b6c3-49a5-b23e-496d6314a0fa\") " pod="kuadrant-system/authorino-54d999fbc7-28znk"
Jul 29 18:01:46.398005 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.397985    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hb4p\" (UniqueName: \"kubernetes.io/projected/45d04d32-b6c3-49a5-b23e-496d6314a0fa-kube-api-access-4hb4p\") pod \"authorino-54d999fbc7-28znk\" (UID: \"45d04d32-b6c3-49a5-b23e-496d6314a0fa\") " pod="kuadrant-system/authorino-54d999fbc7-28znk"
Jul 29 18:01:46.552853 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.552810    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54d999fbc7-28znk"
Jul 29 18:01:46.665813 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.665781    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54d999fbc7-28znk"]
Jul 29 18:01:46.696795 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.696762    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6c88ccc7-sn75f"]
Jul 29 18:01:46.698806 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.698790    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6c88ccc7-sn75f"
Jul 29 18:01:46.701931 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.701905    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 29 18:01:46.703094 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.703074    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54d999fbc7-28znk"]
Jul 29 18:01:46.707839 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.707818    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6c88ccc7-sn75f"]
Jul 29 18:01:46.793504 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.793476    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z6zq\" (UniqueName: \"kubernetes.io/projected/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-kube-api-access-2z6zq\") pod \"authorino-6f6c88ccc7-sn75f\" (UID: \"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94\") " pod="kuadrant-system/authorino-6f6c88ccc7-sn75f"
Jul 29 18:01:46.793828 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.793512    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-tls-cert\") pod \"authorino-6f6c88ccc7-sn75f\" (UID: \"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94\") " pod="kuadrant-system/authorino-6f6c88ccc7-sn75f"
Jul 29 18:01:46.895222 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.895183    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z6zq\" (UniqueName: \"kubernetes.io/projected/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-kube-api-access-2z6zq\") pod \"authorino-6f6c88ccc7-sn75f\" (UID: \"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94\") " pod="kuadrant-system/authorino-6f6c88ccc7-sn75f"
Jul 29 18:01:46.895437 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.895249    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-tls-cert\") pod \"authorino-6f6c88ccc7-sn75f\" (UID: \"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94\") " pod="kuadrant-system/authorino-6f6c88ccc7-sn75f"
Jul 29 18:01:46.901588 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.901558    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-tls-cert\") pod \"authorino-6f6c88ccc7-sn75f\" (UID: \"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94\") " pod="kuadrant-system/authorino-6f6c88ccc7-sn75f"
Jul 29 18:01:46.904153 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:46.904127    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z6zq\" (UniqueName: \"kubernetes.io/projected/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-kube-api-access-2z6zq\") pod \"authorino-6f6c88ccc7-sn75f\" (UID: \"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94\") " pod="kuadrant-system/authorino-6f6c88ccc7-sn75f"
Jul 29 18:01:47.010289 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.010206    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6c88ccc7-sn75f"
Jul 29 18:01:47.132914 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.132892    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6c88ccc7-sn75f"]
Jul 29 18:01:47.134756 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:01:47.134727    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode133fce8_aba9_4b8d_8e86_1f9f55c8cf94.slice/crio-4fe71f5441cc9b5ad1f1df65a31b44fa7fccc01f8fbeaa01ff68fa0d786e73ca WatchSource:0}: Error finding container 4fe71f5441cc9b5ad1f1df65a31b44fa7fccc01f8fbeaa01ff68fa0d786e73ca: Status 404 returned error can't find the container with id 4fe71f5441cc9b5ad1f1df65a31b44fa7fccc01f8fbeaa01ff68fa0d786e73ca
Jul 29 18:01:47.367206 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.367174    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54d999fbc7-28znk" event={"ID":"45d04d32-b6c3-49a5-b23e-496d6314a0fa","Type":"ContainerStarted","Data":"3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7"}
Jul 29 18:01:47.367403 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.367217    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54d999fbc7-28znk" event={"ID":"45d04d32-b6c3-49a5-b23e-496d6314a0fa","Type":"ContainerStarted","Data":"460db4ef10ea78654d77ebf9248deea2c0bff8bdc8108c8e72efd6d101abbe40"}
Jul 29 18:01:47.367403 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.367219    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-54d999fbc7-28znk" podUID="45d04d32-b6c3-49a5-b23e-496d6314a0fa" containerName="authorino" containerID="cri-o://3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7" gracePeriod=30
Jul 29 18:01:47.368910 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.368881    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-khrg7" event={"ID":"484aad4e-0cbf-4f26-9e12-df31b65c4126","Type":"ContainerStarted","Data":"be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55"}
Jul 29 18:01:47.369047 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.368936    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-khrg7" podUID="484aad4e-0cbf-4f26-9e12-df31b65c4126" containerName="authorino" containerID="cri-o://be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55" gracePeriod=30
Jul 29 18:01:47.370250 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.370159    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6c88ccc7-sn75f" event={"ID":"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94","Type":"ContainerStarted","Data":"4fe71f5441cc9b5ad1f1df65a31b44fa7fccc01f8fbeaa01ff68fa0d786e73ca"}
Jul 29 18:01:47.382812 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.382757    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54d999fbc7-28znk" podStartSLOduration=0.922686931 podStartE2EDuration="1.382739118s" totalImagesPullingTime="460.052187ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:47.381812713 +0000 UTC m=+576.890196309" watchObservedRunningTime="2026-07-29 18:01:47.382739118 +0000 UTC m=+576.891122707"
Jul 29 18:01:47.397230 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.397189    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-khrg7" podStartSLOduration=2.034336357 podStartE2EDuration="2.397176318s" totalImagesPullingTime="362.839961ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:47.396444108 +0000 UTC m=+576.904827698" watchObservedRunningTime="2026-07-29 18:01:47.397176318 +0000 UTC m=+576.905559905"
Jul 29 18:01:47.616839 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.616806    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54d999fbc7-28znk"
Jul 29 18:01:47.623907 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.623884    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-khrg7"
Jul 29 18:01:47.701206 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.701183    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6kx7\" (UniqueName: \"kubernetes.io/projected/484aad4e-0cbf-4f26-9e12-df31b65c4126-kube-api-access-v6kx7\") pod \"484aad4e-0cbf-4f26-9e12-df31b65c4126\" (UID: \"484aad4e-0cbf-4f26-9e12-df31b65c4126\") "
Jul 29 18:01:47.701315 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.701221    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hb4p\" (UniqueName: \"kubernetes.io/projected/45d04d32-b6c3-49a5-b23e-496d6314a0fa-kube-api-access-4hb4p\") pod \"45d04d32-b6c3-49a5-b23e-496d6314a0fa\" (UID: \"45d04d32-b6c3-49a5-b23e-496d6314a0fa\") "
Jul 29 18:01:47.703052 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.703022    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/484aad4e-0cbf-4f26-9e12-df31b65c4126-kube-api-access-v6kx7" (OuterVolumeSpecName: "kube-api-access-v6kx7") pod "484aad4e-0cbf-4f26-9e12-df31b65c4126" (UID: "484aad4e-0cbf-4f26-9e12-df31b65c4126"). InnerVolumeSpecName "kube-api-access-v6kx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:01:47.703131 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.703109    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45d04d32-b6c3-49a5-b23e-496d6314a0fa-kube-api-access-4hb4p" (OuterVolumeSpecName: "kube-api-access-4hb4p") pod "45d04d32-b6c3-49a5-b23e-496d6314a0fa" (UID: "45d04d32-b6c3-49a5-b23e-496d6314a0fa"). InnerVolumeSpecName "kube-api-access-4hb4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:01:47.802290 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.802253    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6kx7\" (UniqueName: \"kubernetes.io/projected/484aad4e-0cbf-4f26-9e12-df31b65c4126-kube-api-access-v6kx7\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:01:47.802290 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:47.802283    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hb4p\" (UniqueName: \"kubernetes.io/projected/45d04d32-b6c3-49a5-b23e-496d6314a0fa-kube-api-access-4hb4p\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:01:48.375091 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.375014    2556 generic.go:356] "Generic (PLEG): container finished" podID="45d04d32-b6c3-49a5-b23e-496d6314a0fa" containerID="3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7" exitCode=0
Jul 29 18:01:48.375091 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.375059    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54d999fbc7-28znk"
Jul 29 18:01:48.375275 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.375107    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54d999fbc7-28znk" event={"ID":"45d04d32-b6c3-49a5-b23e-496d6314a0fa","Type":"ContainerDied","Data":"3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7"}
Jul 29 18:01:48.375275 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.375149    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54d999fbc7-28znk" event={"ID":"45d04d32-b6c3-49a5-b23e-496d6314a0fa","Type":"ContainerDied","Data":"460db4ef10ea78654d77ebf9248deea2c0bff8bdc8108c8e72efd6d101abbe40"}
Jul 29 18:01:48.375275 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.375170    2556 scope.go:117] "RemoveContainer" containerID="3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7"
Jul 29 18:01:48.376481 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.376280    2556 generic.go:356] "Generic (PLEG): container finished" podID="484aad4e-0cbf-4f26-9e12-df31b65c4126" containerID="be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55" exitCode=0
Jul 29 18:01:48.376481 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.376321    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-khrg7"
Jul 29 18:01:48.376481 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.376352    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-khrg7" event={"ID":"484aad4e-0cbf-4f26-9e12-df31b65c4126","Type":"ContainerDied","Data":"be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55"}
Jul 29 18:01:48.376481 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.376397    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-khrg7" event={"ID":"484aad4e-0cbf-4f26-9e12-df31b65c4126","Type":"ContainerDied","Data":"07c4ef29e3e7f91bd1789c8745291b883c76ca74ce7b1b692b0416a46fff3e18"}
Jul 29 18:01:48.377985 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.377959    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6c88ccc7-sn75f" event={"ID":"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94","Type":"ContainerStarted","Data":"e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22"}
Jul 29 18:01:48.383910 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.383893    2556 scope.go:117] "RemoveContainer" containerID="3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7"
Jul 29 18:01:48.384412 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:01:48.384161    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7\": container with ID starting with 3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7 not found: ID does not exist" containerID="3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7"
Jul 29 18:01:48.384412 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.384192    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7"} err="failed to get container status \"3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7\": rpc error: code = NotFound desc = could not find container \"3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7\": container with ID starting with 3494b1bb985142db76ec33f8c1be328591170224d9bc178cf35b10a5bb8d79a7 not found: ID does not exist"
Jul 29 18:01:48.384412 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.384214    2556 scope.go:117] "RemoveContainer" containerID="be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55"
Jul 29 18:01:48.391342 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.391326    2556 scope.go:117] "RemoveContainer" containerID="be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55"
Jul 29 18:01:48.391600 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:01:48.391584    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55\": container with ID starting with be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55 not found: ID does not exist" containerID="be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55"
Jul 29 18:01:48.391662 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.391606    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55"} err="failed to get container status \"be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55\": rpc error: code = NotFound desc = could not find container \"be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55\": container with ID starting with be3540a8d9a95cc958216638e7aefcb4bc90c0bc2b431804b8898855c7242d55 not found: ID does not exist"
Jul 29 18:01:48.396736 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.396698    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6f6c88ccc7-sn75f" podStartSLOduration=1.8888742600000001 podStartE2EDuration="2.396686548s" totalImagesPullingTime="507.812288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:48.39490042 +0000 UTC m=+577.903284008" watchObservedRunningTime="2026-07-29 18:01:48.396686548 +0000 UTC m=+577.905070135"
Jul 29 18:01:48.412785 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.412759    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-khrg7"]
Jul 29 18:01:48.419603 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.419582    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-khrg7"]
Jul 29 18:01:48.429815 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.429794    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nxzpp"]
Jul 29 18:01:48.429957 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.429939    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-nxzpp" podUID="969e560e-4a94-4a96-ad93-1249436bd1ba" containerName="authorino" containerID="cri-o://01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959" gracePeriod=30
Jul 29 18:01:48.434665 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.434640    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54d999fbc7-28znk"]
Jul 29 18:01:48.441794 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.441771    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-54d999fbc7-28znk"]
Jul 29 18:01:48.672472 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.672450    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nxzpp"
Jul 29 18:01:48.708497 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.708472    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rd6cd\" (UniqueName: \"kubernetes.io/projected/969e560e-4a94-4a96-ad93-1249436bd1ba-kube-api-access-rd6cd\") pod \"969e560e-4a94-4a96-ad93-1249436bd1ba\" (UID: \"969e560e-4a94-4a96-ad93-1249436bd1ba\") "
Jul 29 18:01:48.710658 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.710633    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969e560e-4a94-4a96-ad93-1249436bd1ba-kube-api-access-rd6cd" (OuterVolumeSpecName: "kube-api-access-rd6cd") pod "969e560e-4a94-4a96-ad93-1249436bd1ba" (UID: "969e560e-4a94-4a96-ad93-1249436bd1ba"). InnerVolumeSpecName "kube-api-access-rd6cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:01:48.809234 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:48.809201    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd6cd\" (UniqueName: \"kubernetes.io/projected/969e560e-4a94-4a96-ad93-1249436bd1ba-kube-api-access-rd6cd\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:01:49.078084 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:49.078007    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45d04d32-b6c3-49a5-b23e-496d6314a0fa" path="/var/lib/kubelet/pods/45d04d32-b6c3-49a5-b23e-496d6314a0fa/volumes"
Jul 29 18:01:49.078322 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:49.078308    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="484aad4e-0cbf-4f26-9e12-df31b65c4126" path="/var/lib/kubelet/pods/484aad4e-0cbf-4f26-9e12-df31b65c4126/volumes"
Jul 29 18:01:49.384743 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:49.384710    2556 generic.go:356] "Generic (PLEG): container finished" podID="969e560e-4a94-4a96-ad93-1249436bd1ba" containerID="01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959" exitCode=0
Jul 29 18:01:49.384905 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:49.384751    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nxzpp"
Jul 29 18:01:49.384905 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:49.384786    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nxzpp" event={"ID":"969e560e-4a94-4a96-ad93-1249436bd1ba","Type":"ContainerDied","Data":"01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959"}
Jul 29 18:01:49.384905 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:49.384823    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nxzpp" event={"ID":"969e560e-4a94-4a96-ad93-1249436bd1ba","Type":"ContainerDied","Data":"0adc939408090b538d7a2f312387e6c6c0ce9c4f1a5cb1b3d8ffa7277207c9ab"}
Jul 29 18:01:49.384905 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:49.384844    2556 scope.go:117] "RemoveContainer" containerID="01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959"
Jul 29 18:01:49.392904 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:49.392880    2556 scope.go:117] "RemoveContainer" containerID="01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959"
Jul 29 18:01:49.393177 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:01:49.393160    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959\": container with ID starting with 01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959 not found: ID does not exist" containerID="01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959"
Jul 29 18:01:49.393236 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:49.393185    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959"} err="failed to get container status \"01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959\": rpc error: code = NotFound desc = could not find container \"01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959\": container with ID starting with 01d76ae054accef1a94cf1372439bf40f89ef76990df1b4b478b919cae2a3959 not found: ID does not exist"
Jul 29 18:01:49.403244 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:49.403221    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nxzpp"]
Jul 29 18:01:49.408492 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:49.408472    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nxzpp"]
Jul 29 18:01:51.078233 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:51.078188    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="969e560e-4a94-4a96-ad93-1249436bd1ba" path="/var/lib/kubelet/pods/969e560e-4a94-4a96-ad93-1249436bd1ba/volumes"
Jul 29 18:01:53.134344 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:53.134314    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:53.247588 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:53.247550    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:53.341486 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:53.341453    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:53.482671 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:53.482591    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:56.273381 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:56.273329    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:01:56.381642 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:01:56.381612    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:02:11.758651 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:02:11.758614    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:02:32.356266 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:02:32.356218    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:02:36.100997 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:02:36.100922    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:02:45.817172 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:02:45.817138    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:02:45.952208 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:02:45.952179    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:02:58.715345 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:02:58.715312    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:03:04.019293 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.019260    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"]
Jul 29 18:03:04.019909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.019661    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969e560e-4a94-4a96-ad93-1249436bd1ba" containerName="authorino"
Jul 29 18:03:04.019909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.019677    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="969e560e-4a94-4a96-ad93-1249436bd1ba" containerName="authorino"
Jul 29 18:03:04.019909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.019701    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45d04d32-b6c3-49a5-b23e-496d6314a0fa" containerName="authorino"
Jul 29 18:03:04.019909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.019709    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="45d04d32-b6c3-49a5-b23e-496d6314a0fa" containerName="authorino"
Jul 29 18:03:04.019909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.019722    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="484aad4e-0cbf-4f26-9e12-df31b65c4126" containerName="authorino"
Jul 29 18:03:04.019909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.019732    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="484aad4e-0cbf-4f26-9e12-df31b65c4126" containerName="authorino"
Jul 29 18:03:04.019909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.019803    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="484aad4e-0cbf-4f26-9e12-df31b65c4126" containerName="authorino"
Jul 29 18:03:04.019909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.019813    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="45d04d32-b6c3-49a5-b23e-496d6314a0fa" containerName="authorino"
Jul 29 18:03:04.019909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.019823    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="969e560e-4a94-4a96-ad93-1249436bd1ba" containerName="authorino"
Jul 29 18:03:04.021740 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.021724    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.024292 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.024271    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-w4f67\""
Jul 29 18:03:04.025528 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.025512    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 18:03:04.025725 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.025693    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 29 18:03:04.025830 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.025765    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 18:03:04.031148 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.031127    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"]
Jul 29 18:03:04.052137 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.052109    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:03:04.085051 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.085020    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.085197 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.085063    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.085197 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.085088    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.085197 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.085161    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbn8c\" (UniqueName: \"kubernetes.io/projected/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-kube-api-access-lbn8c\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.085352 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.085210    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.085352 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.085244    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.186470 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.186440    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.186640 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.186475    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.186702 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.186676    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.186766 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.186717    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.186766 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.186748    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.186862 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.186822    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbn8c\" (UniqueName: \"kubernetes.io/projected/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-kube-api-access-lbn8c\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.186862 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.186851    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.186942 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.186869    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.187115 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.187091    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.190385 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.190332    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.190677 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.190658    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.194620 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.194591    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbn8c\" (UniqueName: \"kubernetes.io/projected/5c1e378d-dd3b-41cc-a7aa-2032dd6101e7-kube-api-access-lbn8c\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg\" (UID: \"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.334194 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.334108    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:04.460851 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.460823    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"]
Jul 29 18:03:04.463295 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:03:04.463266    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c1e378d_dd3b_41cc_a7aa_2032dd6101e7.slice/crio-2ae40aa7270105e8a0e1a2336bf6d55d915099270cc213a1480a14d98ce3827f WatchSource:0}: Error finding container 2ae40aa7270105e8a0e1a2336bf6d55d915099270cc213a1480a14d98ce3827f: Status 404 returned error can't find the container with id 2ae40aa7270105e8a0e1a2336bf6d55d915099270cc213a1480a14d98ce3827f
Jul 29 18:03:04.464590 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.464572    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:03:04.679308 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:04.679269    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg" event={"ID":"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7","Type":"ContainerStarted","Data":"2ae40aa7270105e8a0e1a2336bf6d55d915099270cc213a1480a14d98ce3827f"}
Jul 29 18:03:06.662288 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:06.662256    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:03:06.690965 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:06.690936    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg" event={"ID":"5c1e378d-dd3b-41cc-a7aa-2032dd6101e7","Type":"ContainerStarted","Data":"9f5f041d5cd2b0dcd752bf4d8d0f0d66d80a6fac61a36adf5afb1c0d1f7ca9f5"}
Jul 29 18:03:06.712207 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:06.712160    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg" podStartSLOduration=1.335494555 podStartE2EDuration="2.712145396s" totalImagesPullingTime="1.376650841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:03:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:03:06.710884065 +0000 UTC m=+656.219267656" watchObservedRunningTime="2026-07-29 18:03:06.712145396 +0000 UTC m=+656.220528983"
Jul 29 18:03:07.001006 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:07.000926    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:03:07.079004 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:07.078970    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:03:14.334615 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:14.334577    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:14.335007 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:14.334644    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:14.347104 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:14.347080    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:14.733834 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:14.733807    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg"
Jul 29 18:03:21.750413 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:21.750381    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:03:30.187098 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:30.187063    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:03:30.246762 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:30.246734    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:03:49.175539 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:03:49.175504    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:04:03.607582 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.607501    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"]
Jul 29 18:04:03.611910 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.611889    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.614483 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.614461    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 29 18:04:03.621586 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.621565    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"]
Jul 29 18:04:03.634780 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.634754    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:04:03.752198 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.752164    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.752198 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.752202    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.752403 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.752228    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.752403 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.752291    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm8vx\" (UniqueName: \"kubernetes.io/projected/5517c982-2206-4211-934d-51908a3dc879-kube-api-access-hm8vx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.752403 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.752336    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.752403 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.752389    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5517c982-2206-4211-934d-51908a3dc879-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.853601 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.853569    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.853734 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.853610    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.853734 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.853640    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.853734 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.853672    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm8vx\" (UniqueName: \"kubernetes.io/projected/5517c982-2206-4211-934d-51908a3dc879-kube-api-access-hm8vx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.853734 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.853711    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.853902 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.853751    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5517c982-2206-4211-934d-51908a3dc879-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.854059 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.854035    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.854130 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.854048    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.854187 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.854166    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.855924 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.855901    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5517c982-2206-4211-934d-51908a3dc879-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.856172 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.856156    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5517c982-2206-4211-934d-51908a3dc879-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.862864 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.862794    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm8vx\" (UniqueName: \"kubernetes.io/projected/5517c982-2206-4211-934d-51908a3dc879-kube-api-access-hm8vx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j\" (UID: \"5517c982-2206-4211-934d-51908a3dc879\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:03.921794 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:03.921764    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:04.048040 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:04.047950    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"]
Jul 29 18:04:04.050684 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:04:04.050654    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5517c982_2206_4211_934d_51908a3dc879.slice/crio-061679dd73028891f709151cc0bbc8b8e6d28f724acf20021cc740281c8f7660 WatchSource:0}: Error finding container 061679dd73028891f709151cc0bbc8b8e6d28f724acf20021cc740281c8f7660: Status 404 returned error can't find the container with id 061679dd73028891f709151cc0bbc8b8e6d28f724acf20021cc740281c8f7660
Jul 29 18:04:04.925684 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:04.925650    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j" event={"ID":"5517c982-2206-4211-934d-51908a3dc879","Type":"ContainerStarted","Data":"5003a71004f7297ffd80b10d894d1ebf89907783909c63e2c7c9e3ee068c5f26"}
Jul 29 18:04:04.926038 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:04.925704    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j" event={"ID":"5517c982-2206-4211-934d-51908a3dc879","Type":"ContainerStarted","Data":"061679dd73028891f709151cc0bbc8b8e6d28f724acf20021cc740281c8f7660"}
Jul 29 18:04:04.943234 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:04.943188    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j" podStartSLOduration=1.559112989 podStartE2EDuration="1.94317077s" totalImagesPullingTime="384.057781ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:04:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:04:04.942439405 +0000 UTC m=+714.450822983" watchObservedRunningTime="2026-07-29 18:04:04.94317077 +0000 UTC m=+714.451554359"
Jul 29 18:04:10.148772 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:10.148733    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:04:13.922466 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:13.922430    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:13.922836 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:13.922536    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:13.934557 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:13.934527    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:13.972458 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:13.972429    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j"
Jul 29 18:04:16.720493 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.720459    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"]
Jul 29 18:04:16.723737 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.723717    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.726453 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.726431    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 29 18:04:16.739113 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.739093    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"]
Jul 29 18:04:16.755701 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.755680    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.755795 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.755749    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a6528d-514b-4995-849a-ca898f933b92-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.755795 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.755772    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbcgq\" (UniqueName: \"kubernetes.io/projected/66a6528d-514b-4995-849a-ca898f933b92-kube-api-access-zbcgq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.755909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.755797    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.755909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.755860    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.756316 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.755938    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.763814 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.763793    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:04:16.856974 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.856942    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.856974 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.856978    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.857153 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.856996    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.857153 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.857024    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.857153 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.857087    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a6528d-514b-4995-849a-ca898f933b92-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.857153 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.857114    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbcgq\" (UniqueName: \"kubernetes.io/projected/66a6528d-514b-4995-849a-ca898f933b92-kube-api-access-zbcgq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.857425 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.857403    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.857513 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.857471    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.857571 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.857552    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.859287 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.859269    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66a6528d-514b-4995-849a-ca898f933b92-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.859405 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.859387    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a6528d-514b-4995-849a-ca898f933b92-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:16.864687 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:16.864666    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbcgq\" (UniqueName: \"kubernetes.io/projected/66a6528d-514b-4995-849a-ca898f933b92-kube-api-access-zbcgq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr\" (UID: \"66a6528d-514b-4995-849a-ca898f933b92\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:17.034785 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:17.034704    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:17.166128 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:17.166104    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"]
Jul 29 18:04:17.168749 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:04:17.168720    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66a6528d_514b_4995_849a_ca898f933b92.slice/crio-42691334e3bf628f4c0e7b0ee932b3eb3e0d7215857386ea8dbb5e7d3eaecef9 WatchSource:0}: Error finding container 42691334e3bf628f4c0e7b0ee932b3eb3e0d7215857386ea8dbb5e7d3eaecef9: Status 404 returned error can't find the container with id 42691334e3bf628f4c0e7b0ee932b3eb3e0d7215857386ea8dbb5e7d3eaecef9
Jul 29 18:04:17.978185 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:17.978150    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr" event={"ID":"66a6528d-514b-4995-849a-ca898f933b92","Type":"ContainerStarted","Data":"a086a1c2c803a46076595080fb5f796c22149ce7a19d61edd62da5d2d9f03405"}
Jul 29 18:04:17.978577 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:17.978196    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr" event={"ID":"66a6528d-514b-4995-849a-ca898f933b92","Type":"ContainerStarted","Data":"42691334e3bf628f4c0e7b0ee932b3eb3e0d7215857386ea8dbb5e7d3eaecef9"}
Jul 29 18:04:17.995397 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:17.995332    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr" podStartSLOduration=1.764959554 podStartE2EDuration="1.995318807s" totalImagesPullingTime="230.359253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:04:17.994080743 +0000 UTC m=+727.502464331" watchObservedRunningTime="2026-07-29 18:04:17.995318807 +0000 UTC m=+727.503702395"
Jul 29 18:04:27.035475 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:27.035427    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:27.035475 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:27.035496    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:27.048164 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:27.048135    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:04:28.027753 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:04:28.027721    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr"
Jul 29 18:05:10.836379 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:10.836329    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78c979c496-s7kr6"]
Jul 29 18:05:10.839338 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:10.839322    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78c979c496-s7kr6"
Jul 29 18:05:10.850048 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:10.850024    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78c979c496-s7kr6"]
Jul 29 18:05:10.993519 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:10.993488    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9nqg\" (UniqueName: \"kubernetes.io/projected/a83195f9-3b92-4940-be8e-1a13a58f5464-kube-api-access-q9nqg\") pod \"authorino-78c979c496-s7kr6\" (UID: \"a83195f9-3b92-4940-be8e-1a13a58f5464\") " pod="kuadrant-system/authorino-78c979c496-s7kr6"
Jul 29 18:05:10.993671 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:10.993544    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a83195f9-3b92-4940-be8e-1a13a58f5464-tls-cert\") pod \"authorino-78c979c496-s7kr6\" (UID: \"a83195f9-3b92-4940-be8e-1a13a58f5464\") " pod="kuadrant-system/authorino-78c979c496-s7kr6"
Jul 29 18:05:11.094155 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:11.094077    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a83195f9-3b92-4940-be8e-1a13a58f5464-tls-cert\") pod \"authorino-78c979c496-s7kr6\" (UID: \"a83195f9-3b92-4940-be8e-1a13a58f5464\") " pod="kuadrant-system/authorino-78c979c496-s7kr6"
Jul 29 18:05:11.094155 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:11.094138    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9nqg\" (UniqueName: \"kubernetes.io/projected/a83195f9-3b92-4940-be8e-1a13a58f5464-kube-api-access-q9nqg\") pod \"authorino-78c979c496-s7kr6\" (UID: \"a83195f9-3b92-4940-be8e-1a13a58f5464\") " pod="kuadrant-system/authorino-78c979c496-s7kr6"
Jul 29 18:05:11.096464 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:11.096444    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a83195f9-3b92-4940-be8e-1a13a58f5464-tls-cert\") pod \"authorino-78c979c496-s7kr6\" (UID: \"a83195f9-3b92-4940-be8e-1a13a58f5464\") " pod="kuadrant-system/authorino-78c979c496-s7kr6"
Jul 29 18:05:11.102988 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:11.102964    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9nqg\" (UniqueName: \"kubernetes.io/projected/a83195f9-3b92-4940-be8e-1a13a58f5464-kube-api-access-q9nqg\") pod \"authorino-78c979c496-s7kr6\" (UID: \"a83195f9-3b92-4940-be8e-1a13a58f5464\") " pod="kuadrant-system/authorino-78c979c496-s7kr6"
Jul 29 18:05:11.148614 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:11.148578    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78c979c496-s7kr6"
Jul 29 18:05:11.291183 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:11.291156    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78c979c496-s7kr6"]
Jul 29 18:05:11.292100 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:05:11.292065    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda83195f9_3b92_4940_be8e_1a13a58f5464.slice/crio-1b56807407e98ad06e7a258964e835c18a9ddcdc215ae52cc78e8411140d167c WatchSource:0}: Error finding container 1b56807407e98ad06e7a258964e835c18a9ddcdc215ae52cc78e8411140d167c: Status 404 returned error can't find the container with id 1b56807407e98ad06e7a258964e835c18a9ddcdc215ae52cc78e8411140d167c
Jul 29 18:05:11.296389 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:11.296323    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:05:11.390131 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:11.390104    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:05:12.192121 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.192065    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78c979c496-s7kr6" event={"ID":"a83195f9-3b92-4940-be8e-1a13a58f5464","Type":"ContainerStarted","Data":"c33b52b14bc27fcad9f506d623659261fda76383dd6f3d6fa5782da3019c6fea"}
Jul 29 18:05:12.192527 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.192134    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78c979c496-s7kr6" event={"ID":"a83195f9-3b92-4940-be8e-1a13a58f5464","Type":"ContainerStarted","Data":"1b56807407e98ad06e7a258964e835c18a9ddcdc215ae52cc78e8411140d167c"}
Jul 29 18:05:12.220808 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.220744    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-78c979c496-s7kr6" podStartSLOduration=1.8079128450000002 podStartE2EDuration="2.220721811s" totalImagesPullingTime="412.808966ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:05:12.219287365 +0000 UTC m=+781.727670956" watchObservedRunningTime="2026-07-29 18:05:12.220721811 +0000 UTC m=+781.729105402"
Jul 29 18:05:12.286481 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.286449    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6c88ccc7-sn75f"]
Jul 29 18:05:12.286674 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.286631    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6f6c88ccc7-sn75f" podUID="e133fce8-aba9-4b8d-8e86-1f9f55c8cf94" containerName="authorino" containerID="cri-o://e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22" gracePeriod=30
Jul 29 18:05:12.380966 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.380934    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:05:12.470269 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.470236    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:05:12.541025 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.540996    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6c88ccc7-sn75f"
Jul 29 18:05:12.709428 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.709393    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-tls-cert\") pod \"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94\" (UID: \"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94\") "
Jul 29 18:05:12.709592 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.709443    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2z6zq\" (UniqueName: \"kubernetes.io/projected/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-kube-api-access-2z6zq\") pod \"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94\" (UID: \"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94\") "
Jul 29 18:05:12.711571 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.711535    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-kube-api-access-2z6zq" (OuterVolumeSpecName: "kube-api-access-2z6zq") pod "e133fce8-aba9-4b8d-8e86-1f9f55c8cf94" (UID: "e133fce8-aba9-4b8d-8e86-1f9f55c8cf94"). InnerVolumeSpecName "kube-api-access-2z6zq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:05:12.718790 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.718737    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e133fce8-aba9-4b8d-8e86-1f9f55c8cf94" (UID: "e133fce8-aba9-4b8d-8e86-1f9f55c8cf94"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:05:12.810265 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.810232    2556 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-tls-cert\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:05:12.810265 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:12.810260    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z6zq\" (UniqueName: \"kubernetes.io/projected/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94-kube-api-access-2z6zq\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:05:13.196895 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:13.196862    2556 generic.go:356] "Generic (PLEG): container finished" podID="e133fce8-aba9-4b8d-8e86-1f9f55c8cf94" containerID="e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22" exitCode=0
Jul 29 18:05:13.197343 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:13.196918    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6c88ccc7-sn75f"
Jul 29 18:05:13.197343 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:13.196948    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6c88ccc7-sn75f" event={"ID":"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94","Type":"ContainerDied","Data":"e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22"}
Jul 29 18:05:13.197343 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:13.197000    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6c88ccc7-sn75f" event={"ID":"e133fce8-aba9-4b8d-8e86-1f9f55c8cf94","Type":"ContainerDied","Data":"4fe71f5441cc9b5ad1f1df65a31b44fa7fccc01f8fbeaa01ff68fa0d786e73ca"}
Jul 29 18:05:13.197343 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:13.197025    2556 scope.go:117] "RemoveContainer" containerID="e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22"
Jul 29 18:05:13.206721 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:13.206703    2556 scope.go:117] "RemoveContainer" containerID="e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22"
Jul 29 18:05:13.206984 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:05:13.206967    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22\": container with ID starting with e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22 not found: ID does not exist" containerID="e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22"
Jul 29 18:05:13.207030 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:13.206992    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22"} err="failed to get container status \"e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22\": rpc error: code = NotFound desc = could not find container \"e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22\": container with ID starting with e41d9b446efc717e50c04ce3881be8369faf58cb5b73baccce9e78b52269fc22 not found: ID does not exist"
Jul 29 18:05:13.211273 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:13.211250    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6c88ccc7-sn75f"]
Jul 29 18:05:13.215897 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:13.215871    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6c88ccc7-sn75f"]
Jul 29 18:05:15.078291 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:15.078259    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e133fce8-aba9-4b8d-8e86-1f9f55c8cf94" path="/var/lib/kubelet/pods/e133fce8-aba9-4b8d-8e86-1f9f55c8cf94/volumes"
Jul 29 18:05:54.462352 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:05:54.462313    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:06:43.149775 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:06:43.149740    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:06:43.286771 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:06:43.286736    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:07:16.724449 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:07:16.724352    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:07:17.962520 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:07:17.962488    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:07:53.336594 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:07:53.336550    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:08:26.857883 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:08:26.857848    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:08:26.955106 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:08:26.955077    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:08:27.091103 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:08:27.091076    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:08:32.712879 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:08:32.712840    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:08:32.842655 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:08:32.842617    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:08:32.940230 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:08:32.940194    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:08:33.039996 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:08:33.039902    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:08:38.530108 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:08:38.530013    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:08:38.599145 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:08:38.599117    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:10:38.634759 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:10:38.634681    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:10:45.692609 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:10:45.692572    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:15:55.683378 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:15:55.683342    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:16:40.916830 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:16:40.916795    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:17:28.209841 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:17:28.209811    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:17:38.304238 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:17:38.304161    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:18:36.140080 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:18:36.140045    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:18:43.175562 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:18:43.175530    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:18:43.276340 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:18:43.276310    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:18:55.583727 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:18:55.583678    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:18:55.683804 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:18:55.683771    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:19:10.927781 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:10.927702    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"]
Jul 29 18:19:10.928213 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:10.928047    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e133fce8-aba9-4b8d-8e86-1f9f55c8cf94" containerName="authorino"
Jul 29 18:19:10.928213 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:10.928065    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e133fce8-aba9-4b8d-8e86-1f9f55c8cf94" containerName="authorino"
Jul 29 18:19:10.928213 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:10.928115    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e133fce8-aba9-4b8d-8e86-1f9f55c8cf94" containerName="authorino"
Jul 29 18:19:10.930962 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:10.930947    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:10.933562 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:10.933542    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-8faee3e7-kserve-self-signed-certs\""
Jul 29 18:19:10.941660 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:10.941625    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"]
Jul 29 18:19:10.967210 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:10.967176    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:19:11.067572 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.067541    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-tmp-dir\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.067761 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.067580    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/863cbb9b-b7c6-4829-b469-16bce21921f3-tls-certs\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.067761 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.067602    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-home\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.067761 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.067660    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsff8\" (UniqueName: \"kubernetes.io/projected/863cbb9b-b7c6-4829-b469-16bce21921f3-kube-api-access-bsff8\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.067761 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.067737    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-dshm\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.067935 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.067769    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-model-cache\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.168900 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.168861    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-tmp-dir\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.168900 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.168904    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/863cbb9b-b7c6-4829-b469-16bce21921f3-tls-certs\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.169144 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.168929    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-home\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.169144 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.168950    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsff8\" (UniqueName: \"kubernetes.io/projected/863cbb9b-b7c6-4829-b469-16bce21921f3-kube-api-access-bsff8\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.169144 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.168988    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-dshm\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.169144 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.169007    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-model-cache\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.169356 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.169257    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-tmp-dir\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.169356 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.169330    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-home\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.169471 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.169438    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-model-cache\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.171252 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.171230    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-dshm\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.171488 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.171469    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/863cbb9b-b7c6-4829-b469-16bce21921f3-tls-certs\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.177220 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.177196    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsff8\" (UniqueName: \"kubernetes.io/projected/863cbb9b-b7c6-4829-b469-16bce21921f3-kube-api-access-bsff8\") pod \"e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") " pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.240228 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.240141    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:11.370716 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.370681    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"]
Jul 29 18:19:11.374790 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:19:11.374763    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod863cbb9b_b7c6_4829_b469_16bce21921f3.slice/crio-69ee3afab967125849575180e754f81ad42160247f68ab3c39e196a72aff489c WatchSource:0}: Error finding container 69ee3afab967125849575180e754f81ad42160247f68ab3c39e196a72aff489c: Status 404 returned error can't find the container with id 69ee3afab967125849575180e754f81ad42160247f68ab3c39e196a72aff489c
Jul 29 18:19:11.552008 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.551909    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm" event={"ID":"863cbb9b-b7c6-4829-b469-16bce21921f3","Type":"ContainerStarted","Data":"209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1"}
Jul 29 18:19:11.552008 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.551957    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm" event={"ID":"863cbb9b-b7c6-4829-b469-16bce21921f3","Type":"ContainerStarted","Data":"69ee3afab967125849575180e754f81ad42160247f68ab3c39e196a72aff489c"}
Jul 29 18:19:11.568109 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:11.568061    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm" podStartSLOduration=1.568045066 podStartE2EDuration="1.568045066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:19:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:19:11.567240974 +0000 UTC m=+1621.075624561" watchObservedRunningTime="2026-07-29 18:19:11.568045066 +0000 UTC m=+1621.076428653"
Jul 29 18:19:21.240577 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:21.240526    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:21.241113 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:21.240615    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:21.253004 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:21.252979    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:21.606436 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:21.606339    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:19:52.764019 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:52.763976    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:19:53.681685 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:53.681653    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"]
Jul 29 18:19:53.711198 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:19:53.711170    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:20:08.700150 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:08.700095    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm" podUID="863cbb9b-b7c6-4829-b469-16bce21921f3" containerName="main" containerID="cri-o://209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1" gracePeriod=45
Jul 29 18:20:08.948611 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:08.948589    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:20:09.059182 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059097    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsff8\" (UniqueName: \"kubernetes.io/projected/863cbb9b-b7c6-4829-b469-16bce21921f3-kube-api-access-bsff8\") pod \"863cbb9b-b7c6-4829-b469-16bce21921f3\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") "
Jul 29 18:20:09.059182 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059167    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-home\") pod \"863cbb9b-b7c6-4829-b469-16bce21921f3\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") "
Jul 29 18:20:09.059388 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059245    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-tmp-dir\") pod \"863cbb9b-b7c6-4829-b469-16bce21921f3\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") "
Jul 29 18:20:09.059388 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059292    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-dshm\") pod \"863cbb9b-b7c6-4829-b469-16bce21921f3\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") "
Jul 29 18:20:09.059388 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059327    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/863cbb9b-b7c6-4829-b469-16bce21921f3-tls-certs\") pod \"863cbb9b-b7c6-4829-b469-16bce21921f3\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") "
Jul 29 18:20:09.059388 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059349    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-model-cache\") pod \"863cbb9b-b7c6-4829-b469-16bce21921f3\" (UID: \"863cbb9b-b7c6-4829-b469-16bce21921f3\") "
Jul 29 18:20:09.059586 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059444    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-home" (OuterVolumeSpecName: "home") pod "863cbb9b-b7c6-4829-b469-16bce21921f3" (UID: "863cbb9b-b7c6-4829-b469-16bce21921f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:20:09.059586 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059550    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "863cbb9b-b7c6-4829-b469-16bce21921f3" (UID: "863cbb9b-b7c6-4829-b469-16bce21921f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:20:09.059684 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059621    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-model-cache" (OuterVolumeSpecName: "model-cache") pod "863cbb9b-b7c6-4829-b469-16bce21921f3" (UID: "863cbb9b-b7c6-4829-b469-16bce21921f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:20:09.059735 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059714    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-tmp-dir\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:20:09.059735 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059731    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-model-cache\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:20:09.059835 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.059740    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-home\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:20:09.061439 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.061413    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-dshm" (OuterVolumeSpecName: "dshm") pod "863cbb9b-b7c6-4829-b469-16bce21921f3" (UID: "863cbb9b-b7c6-4829-b469-16bce21921f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:20:09.061569 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.061535    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/863cbb9b-b7c6-4829-b469-16bce21921f3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "863cbb9b-b7c6-4829-b469-16bce21921f3" (UID: "863cbb9b-b7c6-4829-b469-16bce21921f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:20:09.061569 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.061552    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/863cbb9b-b7c6-4829-b469-16bce21921f3-kube-api-access-bsff8" (OuterVolumeSpecName: "kube-api-access-bsff8") pod "863cbb9b-b7c6-4829-b469-16bce21921f3" (UID: "863cbb9b-b7c6-4829-b469-16bce21921f3"). InnerVolumeSpecName "kube-api-access-bsff8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:20:09.160106 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.160079    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/863cbb9b-b7c6-4829-b469-16bce21921f3-dshm\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:20:09.160106 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.160105    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/863cbb9b-b7c6-4829-b469-16bce21921f3-tls-certs\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:20:09.160281 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.160116    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsff8\" (UniqueName: \"kubernetes.io/projected/863cbb9b-b7c6-4829-b469-16bce21921f3-kube-api-access-bsff8\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:20:09.788520 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.788486    2556 generic.go:356] "Generic (PLEG): container finished" podID="863cbb9b-b7c6-4829-b469-16bce21921f3" containerID="209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1" exitCode=0
Jul 29 18:20:09.788966 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.788527    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"
Jul 29 18:20:09.788966 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.788567    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm" event={"ID":"863cbb9b-b7c6-4829-b469-16bce21921f3","Type":"ContainerDied","Data":"209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1"}
Jul 29 18:20:09.788966 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.788610    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm" event={"ID":"863cbb9b-b7c6-4829-b469-16bce21921f3","Type":"ContainerDied","Data":"69ee3afab967125849575180e754f81ad42160247f68ab3c39e196a72aff489c"}
Jul 29 18:20:09.788966 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.788627    2556 scope.go:117] "RemoveContainer" containerID="209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1"
Jul 29 18:20:09.796585 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.796562    2556 scope.go:117] "RemoveContainer" containerID="209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1"
Jul 29 18:20:09.796830 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:20:09.796812    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1\": container with ID starting with 209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1 not found: ID does not exist" containerID="209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1"
Jul 29 18:20:09.796892 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.796838    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1"} err="failed to get container status \"209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1\": rpc error: code = NotFound desc = could not find container \"209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1\": container with ID starting with 209616f1b16a281e402da4f4ff8de5287bd6ea5559d954470dd53c539d8aa0a1 not found: ID does not exist"
Jul 29 18:20:09.804981 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.804958    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"]
Jul 29 18:20:09.806713 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:09.806693    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-8faee3e7-kserve-59ff44bb95-sz5rm"]
Jul 29 18:20:11.077678 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:20:11.077643    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="863cbb9b-b7c6-4829-b469-16bce21921f3" path="/var/lib/kubelet/pods/863cbb9b-b7c6-4829-b469-16bce21921f3/volumes"
Jul 29 18:23:47.265323 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:23:47.265220    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:24.851489 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:24.851452    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:28.084917 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:28.084875    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:28.183954 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:28.183917    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:28.610459 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:28.610428    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:34.539777 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:34.539743    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:34.619504 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:34.619472    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:34.837069 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:34.836992    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:35.360608 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:35.360575    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:35.460506 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:35.460460    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:37.502930 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:37.502898    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:37.601483 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:37.601453    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:37.674831 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:37.674796    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:38.183982 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:38.183944    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:24:38.283010 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:24:38.282971    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:03.085262 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.085227    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"]
Jul 29 18:25:03.087706 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.085517    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="863cbb9b-b7c6-4829-b469-16bce21921f3" containerName="main"
Jul 29 18:25:03.087706 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.085528    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="863cbb9b-b7c6-4829-b469-16bce21921f3" containerName="main"
Jul 29 18:25:03.087706 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.085584    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="863cbb9b-b7c6-4829-b469-16bce21921f3" containerName="main"
Jul 29 18:25:03.088596 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.088580    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"
Jul 29 18:25:03.091058 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.091034    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 18:25:03.091194 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.091085    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:25:03.096689 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.096637    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"]
Jul 29 18:25:03.102766 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.102733    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6f95dbc-58d0-4599-9dac-b6b4b88a9032-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0174a53b-f9npl\" (UID: \"d6f95dbc-58d0-4599-9dac-b6b4b88a9032\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"
Jul 29 18:25:03.203922 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.203883    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6f95dbc-58d0-4599-9dac-b6b4b88a9032-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0174a53b-f9npl\" (UID: \"d6f95dbc-58d0-4599-9dac-b6b4b88a9032\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"
Jul 29 18:25:03.204544 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.204522    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6f95dbc-58d0-4599-9dac-b6b4b88a9032-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0174a53b-f9npl\" (UID: \"d6f95dbc-58d0-4599-9dac-b6b4b88a9032\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"
Jul 29 18:25:03.399971 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.399909    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"
Jul 29 18:25:03.530937 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.530901    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"]
Jul 29 18:25:03.536849 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:25:03.536819    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f95dbc_58d0_4599_9dac_b6b4b88a9032.slice/crio-d8ad38fa64cff5201402e8c8c5115e8bd14fe1525785bb43470b873b6432cd4d WatchSource:0}: Error finding container d8ad38fa64cff5201402e8c8c5115e8bd14fe1525785bb43470b873b6432cd4d: Status 404 returned error can't find the container with id d8ad38fa64cff5201402e8c8c5115e8bd14fe1525785bb43470b873b6432cd4d
Jul 29 18:25:03.538948 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.538933    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:25:03.966209 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:03.966174    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl" event={"ID":"d6f95dbc-58d0-4599-9dac-b6b4b88a9032","Type":"ContainerStarted","Data":"d8ad38fa64cff5201402e8c8c5115e8bd14fe1525785bb43470b873b6432cd4d"}
Jul 29 18:25:06.981351 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:06.981317    2556 generic.go:356] "Generic (PLEG): container finished" podID="d6f95dbc-58d0-4599-9dac-b6b4b88a9032" containerID="b1600ff3ba33d28cf048b3f36bdf9c4b85ae4947ca95acb2e0bb3ae850dedefb" exitCode=0
Jul 29 18:25:06.981736 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:06.981357    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl" event={"ID":"d6f95dbc-58d0-4599-9dac-b6b4b88a9032","Type":"ContainerDied","Data":"b1600ff3ba33d28cf048b3f36bdf9c4b85ae4947ca95acb2e0bb3ae850dedefb"}
Jul 29 18:25:08.113793 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:08.113767    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"
Jul 29 18:25:08.147998 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:08.147968    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6f95dbc-58d0-4599-9dac-b6b4b88a9032-maas-api-service-ca\") pod \"d6f95dbc-58d0-4599-9dac-b6b4b88a9032\" (UID: \"d6f95dbc-58d0-4599-9dac-b6b4b88a9032\") "
Jul 29 18:25:08.148304 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:08.148283    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6f95dbc-58d0-4599-9dac-b6b4b88a9032-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d6f95dbc-58d0-4599-9dac-b6b4b88a9032" (UID: "d6f95dbc-58d0-4599-9dac-b6b4b88a9032"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:25:08.249153 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:08.249086    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6f95dbc-58d0-4599-9dac-b6b4b88a9032-maas-api-service-ca\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:25:08.991015 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:08.990986    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"
Jul 29 18:25:08.991185 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:08.991017    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl" event={"ID":"d6f95dbc-58d0-4599-9dac-b6b4b88a9032","Type":"ContainerDied","Data":"d8ad38fa64cff5201402e8c8c5115e8bd14fe1525785bb43470b873b6432cd4d"}
Jul 29 18:25:08.991185 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:08.991070    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8ad38fa64cff5201402e8c8c5115e8bd14fe1525785bb43470b873b6432cd4d"
Jul 29 18:25:18.408936 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:18.408900    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:18.509060 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:18.509023    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:18.597568 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:18.597529    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"]
Jul 29 18:25:18.599965 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:18.599938    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0174a53b-f9npl"]
Jul 29 18:25:18.601918 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:18.601895    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:18.681522 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:18.681437    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:18.747423 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:18.747388    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:18.979687 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:18.979606    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:19.078662 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:19.078627    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6f95dbc-58d0-4599-9dac-b6b4b88a9032" path="/var/lib/kubelet/pods/d6f95dbc-58d0-4599-9dac-b6b4b88a9032/volumes"
Jul 29 18:25:19.258783 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:19.258698    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:19.322791 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:19.322757    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:19.468130 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:19.468091    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:19.764618 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:19.764588    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:19.969304 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:19.969273    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:21.928632 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:21.928602    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:22.033213 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:22.033178    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:22.379924 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:22.379880    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:22.653111 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:22.653027    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:22.753886 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:22.753851    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:46.924685 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:46.924650    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"]
Jul 29 18:25:46.925072 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:46.924927    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6f95dbc-58d0-4599-9dac-b6b4b88a9032" containerName="revoke-keys"
Jul 29 18:25:46.925072 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:46.924937    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6f95dbc-58d0-4599-9dac-b6b4b88a9032" containerName="revoke-keys"
Jul 29 18:25:46.925072 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:46.924996    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6f95dbc-58d0-4599-9dac-b6b4b88a9032" containerName="revoke-keys"
Jul 29 18:25:46.927787 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:46.927763    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"
Jul 29 18:25:46.934620 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:46.934599    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:25:46.935520 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:46.935502    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 18:25:46.942438 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:46.942412    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"]
Jul 29 18:25:47.071004 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:47.070974    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/175ff19e-2373-4ae5-b4ea-7970d16a6ff7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98\" (UID: \"175ff19e-2373-4ae5-b4ea-7970d16a6ff7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"
Jul 29 18:25:47.171631 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:47.171596    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/175ff19e-2373-4ae5-b4ea-7970d16a6ff7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98\" (UID: \"175ff19e-2373-4ae5-b4ea-7970d16a6ff7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"
Jul 29 18:25:47.172186 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:47.172168    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/175ff19e-2373-4ae5-b4ea-7970d16a6ff7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98\" (UID: \"175ff19e-2373-4ae5-b4ea-7970d16a6ff7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"
Jul 29 18:25:47.237267 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:47.237193    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"
Jul 29 18:25:47.366420 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:47.366395    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"]
Jul 29 18:25:47.368726 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:25:47.368698    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod175ff19e_2373_4ae5_b4ea_7970d16a6ff7.slice/crio-cc66eff03c6466812323f2735ee9c08045cf83bc351c0a06f0af91bd9f6dd995 WatchSource:0}: Error finding container cc66eff03c6466812323f2735ee9c08045cf83bc351c0a06f0af91bd9f6dd995: Status 404 returned error can't find the container with id cc66eff03c6466812323f2735ee9c08045cf83bc351c0a06f0af91bd9f6dd995
Jul 29 18:25:48.144991 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:48.144960    2556 generic.go:356] "Generic (PLEG): container finished" podID="175ff19e-2373-4ae5-b4ea-7970d16a6ff7" containerID="5ce01308cd0b6cf9984b955b36311204c13ad8ce454abd67fb70ecfb6bb46d4d" exitCode=0
Jul 29 18:25:48.145444 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:48.145024    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98" event={"ID":"175ff19e-2373-4ae5-b4ea-7970d16a6ff7","Type":"ContainerDied","Data":"5ce01308cd0b6cf9984b955b36311204c13ad8ce454abd67fb70ecfb6bb46d4d"}
Jul 29 18:25:48.145444 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:48.145058    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98" event={"ID":"175ff19e-2373-4ae5-b4ea-7970d16a6ff7","Type":"ContainerStarted","Data":"cc66eff03c6466812323f2735ee9c08045cf83bc351c0a06f0af91bd9f6dd995"}
Jul 29 18:25:49.275818 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:49.275795    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"
Jul 29 18:25:49.287042 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:49.287017    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/175ff19e-2373-4ae5-b4ea-7970d16a6ff7-maas-api-service-ca\") pod \"175ff19e-2373-4ae5-b4ea-7970d16a6ff7\" (UID: \"175ff19e-2373-4ae5-b4ea-7970d16a6ff7\") "
Jul 29 18:25:49.287320 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:49.287298    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/175ff19e-2373-4ae5-b4ea-7970d16a6ff7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "175ff19e-2373-4ae5-b4ea-7970d16a6ff7" (UID: "175ff19e-2373-4ae5-b4ea-7970d16a6ff7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:25:49.388292 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:49.388263    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/175ff19e-2373-4ae5-b4ea-7970d16a6ff7-maas-api-service-ca\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:25:50.154684 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:50.154662    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"
Jul 29 18:25:50.154846 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:50.154701    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98" event={"ID":"175ff19e-2373-4ae5-b4ea-7970d16a6ff7","Type":"ContainerDied","Data":"cc66eff03c6466812323f2735ee9c08045cf83bc351c0a06f0af91bd9f6dd995"}
Jul 29 18:25:50.154846 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:50.154741    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc66eff03c6466812323f2735ee9c08045cf83bc351c0a06f0af91bd9f6dd995"
Jul 29 18:25:52.589026 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:52.588992    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:52.689008 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:52.688979    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:52.757808 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:52.757774    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"]
Jul 29 18:25:52.760564 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:52.760541    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa2af080-ksz98"]
Jul 29 18:25:52.762208 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:52.762186    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:52.828303 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:52.828251    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:52.926904 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:52.926866    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:53.079429 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:53.079394    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="175ff19e-2373-4ae5-b4ea-7970d16a6ff7" path="/var/lib/kubelet/pods/175ff19e-2373-4ae5-b4ea-7970d16a6ff7/volumes"
Jul 29 18:25:53.140090 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:53.140059    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:53.366493 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:53.366415    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:53.692773 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:53.692741    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:53.986114 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:53.986030    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:58.473717 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:58.473686    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:58.576238 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:58.576204    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:58.658020 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:58.657983    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:58.764958 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:58.764874    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:59.277840 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:59.277809    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:25:59.393112 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:25:59.393070    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:29.246425 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:29.246386    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:29.346713 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:29.346669    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:29.415633 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:29.415603    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:30.566941 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:30.566908    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:30.816399 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:30.816343    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:31.048026 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:31.047987    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:31.478457 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:31.478407    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:31.592033 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:31.591998    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:33.660492 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:33.660384    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:33.756918 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:33.756879    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:33.952649 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:33.952569    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:34.139990 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:34.139955    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:34.513509 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:34.513474    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:34.612463 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:34.612425    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:36.751768 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:36.751726    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:36.819560 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:36.819513    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:26:58.537606 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.537570    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"]
Jul 29 18:26:58.538006 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.537859    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="175ff19e-2373-4ae5-b4ea-7970d16a6ff7" containerName="revoke-keys"
Jul 29 18:26:58.538006 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.537870    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="175ff19e-2373-4ae5-b4ea-7970d16a6ff7" containerName="revoke-keys"
Jul 29 18:26:58.538006 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.537930    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="175ff19e-2373-4ae5-b4ea-7970d16a6ff7" containerName="revoke-keys"
Jul 29 18:26:58.540821 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.540804    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"
Jul 29 18:26:58.543458 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.543438    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qrbhl\""
Jul 29 18:26:58.543560 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.543462    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 18:26:58.548229 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.547805    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"]
Jul 29 18:26:58.629098 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.629060    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2aae82c3-711d-4d89-9c3a-98633dea8c12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5\" (UID: \"2aae82c3-711d-4d89-9c3a-98633dea8c12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"
Jul 29 18:26:58.730293 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.730263    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2aae82c3-711d-4d89-9c3a-98633dea8c12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5\" (UID: \"2aae82c3-711d-4d89-9c3a-98633dea8c12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"
Jul 29 18:26:58.730889 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.730869    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2aae82c3-711d-4d89-9c3a-98633dea8c12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5\" (UID: \"2aae82c3-711d-4d89-9c3a-98633dea8c12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"
Jul 29 18:26:58.852130 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.852038    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"
Jul 29 18:26:58.977290 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:58.977266    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"]
Jul 29 18:26:58.979271 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:26:58.979241    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aae82c3_711d_4d89_9c3a_98633dea8c12.slice/crio-2beaeafc51e111ad483105fd3b0969046e93cdbe7a498180f607f0a8020cab5c WatchSource:0}: Error finding container 2beaeafc51e111ad483105fd3b0969046e93cdbe7a498180f607f0a8020cab5c: Status 404 returned error can't find the container with id 2beaeafc51e111ad483105fd3b0969046e93cdbe7a498180f607f0a8020cab5c
Jul 29 18:26:59.449562 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:59.449529    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5" event={"ID":"2aae82c3-711d-4d89-9c3a-98633dea8c12","Type":"ContainerStarted","Data":"c622fec66cbf359302ea45ebc421d0ea60baadb858b1de490e35b993b8200030"}
Jul 29 18:26:59.449693 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:59.449587    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5" event={"ID":"2aae82c3-711d-4d89-9c3a-98633dea8c12","Type":"ContainerStarted","Data":"2beaeafc51e111ad483105fd3b0969046e93cdbe7a498180f607f0a8020cab5c"}
Jul 29 18:26:59.464818 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:26:59.464750    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5" podStartSLOduration=1.464731001 podStartE2EDuration="1.464731001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:26:59.463729459 +0000 UTC m=+2088.972113039" watchObservedRunningTime="2026-07-29 18:26:59.464731001 +0000 UTC m=+2088.973114590"
Jul 29 18:27:00.454707 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:00.454673    2556 generic.go:356] "Generic (PLEG): container finished" podID="2aae82c3-711d-4d89-9c3a-98633dea8c12" containerID="c622fec66cbf359302ea45ebc421d0ea60baadb858b1de490e35b993b8200030" exitCode=0
Jul 29 18:27:00.455259 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:00.454762    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5" event={"ID":"2aae82c3-711d-4d89-9c3a-98633dea8c12","Type":"ContainerDied","Data":"c622fec66cbf359302ea45ebc421d0ea60baadb858b1de490e35b993b8200030"}
Jul 29 18:27:01.583673 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:01.583652    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"
Jul 29 18:27:01.653770 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:01.653738    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2aae82c3-711d-4d89-9c3a-98633dea8c12-maas-api-service-ca\") pod \"2aae82c3-711d-4d89-9c3a-98633dea8c12\" (UID: \"2aae82c3-711d-4d89-9c3a-98633dea8c12\") "
Jul 29 18:27:01.654078 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:01.654058    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2aae82c3-711d-4d89-9c3a-98633dea8c12-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2aae82c3-711d-4d89-9c3a-98633dea8c12" (UID: "2aae82c3-711d-4d89-9c3a-98633dea8c12"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:27:01.754939 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:01.754864    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2aae82c3-711d-4d89-9c3a-98633dea8c12-maas-api-service-ca\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:27:02.463906 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:02.463875    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"
Jul 29 18:27:02.463906 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:02.463882    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5" event={"ID":"2aae82c3-711d-4d89-9c3a-98633dea8c12","Type":"ContainerDied","Data":"2beaeafc51e111ad483105fd3b0969046e93cdbe7a498180f607f0a8020cab5c"}
Jul 29 18:27:02.464105 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:02.463923    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2beaeafc51e111ad483105fd3b0969046e93cdbe7a498180f607f0a8020cab5c"
Jul 29 18:27:04.323931 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:04.323889    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:04.395612 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:04.395578    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:04.458372 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:04.458326    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:04.490218 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:04.490182    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"]
Jul 29 18:27:04.493481 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:04.493454    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-eb3353f1-p5pg5"]
Jul 29 18:27:04.521248 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:04.521221    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:04.640438 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:04.640399    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:04.882604 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:04.882573    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:05.079288 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:05.079206    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2aae82c3-711d-4d89-9c3a-98633dea8c12" path="/var/lib/kubelet/pods/2aae82c3-711d-4d89-9c3a-98633dea8c12/volumes"
Jul 29 18:27:05.354348 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:05.354255    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:05.452632 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:05.452590    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:05.552681 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:05.552647    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:27.514218 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:27.514188    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:30.825182 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:30.825150    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:33.483074 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:33.483035    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:33.596540 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:33.596504    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:33.750225 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:33.750129    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:33.895860 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:33.895824    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:27:34.239647 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:27:34.239612    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:00.280695 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:00.280657    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:06.503194 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:06.503103    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:06.569599 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:06.569562    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:06.651510 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:06.651477    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:06.750322 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:06.750290    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:22.630250 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:22.630207    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:28.373823 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:28.373770    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:28.443597 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:28.443566    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:28.537715 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:28.537680    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:39.109278 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:39.109244    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:50.176741 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:50.176702    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:57.199992 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:57.199940    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:57.274278 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:57.274222    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:57.358102 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:57.358067    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:57.454911 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:57.454822    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:28:57.555746 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:28:57.555713    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:00.236471 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:00.236417    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:06.039716 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:06.039626    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:06.106099 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:06.106067    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:06.854661 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:06.854618    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:07.223535 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:07.223503    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:07.406628 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:07.406581    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:10.723034 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:10.722997    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:10.819379 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:10.819334    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:10.898176 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:10.898139    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:11.133659 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:11.133632    2556 scope.go:117] "RemoveContainer" containerID="b1600ff3ba33d28cf048b3f36bdf9c4b85ae4947ca95acb2e0bb3ae850dedefb"
Jul 29 18:31:11.286601 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:11.286568    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:11.422300 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:11.422220    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:11.516908 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:11.516878    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:35.807075 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:35.807034    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:36.480055 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:36.480020    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:36.574096 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:36.574064    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:36.951597 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:36.951559    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:39.639084 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:39.639044    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:39.739602 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:39.739572    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:42.596703 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:42.596661    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:42.670295 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:42.670259    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:53.461487 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:53.461389    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:53.564791 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:53.564753    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:53.647209 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:53.647170    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:53.713996 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:53.713918    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:31:54.041608 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:31:54.041517    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:00.009590 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:00.009556    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:00.075547 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:00.075517    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:00.305419 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:00.305320    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:00.540463 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:00.540432    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:00.666209 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:00.666173    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:00.767299 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:00.767260    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:01.014430 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:01.014341    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:06.850064 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:06.850030    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:06.946051 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:06.946012    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:07.048542 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:07.048510    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:07.274687 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:07.274654    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:11.150968 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:11.150937    2556 scope.go:117] "RemoveContainer" containerID="5ce01308cd0b6cf9984b955b36311204c13ad8ce454abd67fb70ecfb6bb46d4d"
Jul 29 18:32:13.220108 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:13.220069    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:13.321680 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:13.321630    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:26.746252 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:26.746211    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:32:47.612532 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:32:47.612446    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:33:11.169711 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:33:11.169676    2556 scope.go:117] "RemoveContainer" containerID="c622fec66cbf359302ea45ebc421d0ea60baadb858b1de490e35b993b8200030"
Jul 29 18:33:18.676725 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:33:18.676676    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:33:39.641459 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:33:39.641417    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:34:16.021958 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:16.021874    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:34:17.274192 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:17.274150    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:34:19.671337 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:19.671305    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:34:19.768652 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:19.768614    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:34:19.840379 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:19.840341    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:34:20.195150 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:20.195109    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:34:20.466203 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:20.466115    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:34:20.563739 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:20.563681    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:34:44.707819 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:44.707778    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:34:56.423464 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.423433    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"]
Jul 29 18:34:56.423915 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.423715    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aae82c3-711d-4d89-9c3a-98633dea8c12" containerName="revoke-keys"
Jul 29 18:34:56.423915 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.423725    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aae82c3-711d-4d89-9c3a-98633dea8c12" containerName="revoke-keys"
Jul 29 18:34:56.423915 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.423789    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2aae82c3-711d-4d89-9c3a-98633dea8c12" containerName="revoke-keys"
Jul 29 18:34:56.426576 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.426559    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.430283 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.430254    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8be27fe2\"/\"e2e-lifecycle-model-8be27fe2-kserve-self-signed-certs\""
Jul 29 18:34:56.430283 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.430275    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8be27fe2\"/\"openshift-service-ca.crt\""
Jul 29 18:34:56.430476 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.430312    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8be27fe2\"/\"default-dockercfg-bxbdv\""
Jul 29 18:34:56.430476 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.430345    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8be27fe2\"/\"kube-root-ca.crt\""
Jul 29 18:34:56.437131 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.437110    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"]
Jul 29 18:34:56.451857 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.451834    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:34:56.517636 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.517607    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-tmp-dir\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.517636 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.517637    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39661479-8596-4419-a06f-6d498558ad20-tls-certs\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.517792 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.517667    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-dshm\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.517792 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.517684    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-home\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.517792 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.517732    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-model-cache\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.517792 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.517786    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpnll\" (UniqueName: \"kubernetes.io/projected/39661479-8596-4419-a06f-6d498558ad20-kube-api-access-dpnll\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.618183 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.618145    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-tmp-dir\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.618183 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.618177    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39661479-8596-4419-a06f-6d498558ad20-tls-certs\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.618458 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.618212    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-dshm\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.618458 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.618227    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-home\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.618458 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.618246    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-model-cache\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.618458 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.618414    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpnll\" (UniqueName: \"kubernetes.io/projected/39661479-8596-4419-a06f-6d498558ad20-kube-api-access-dpnll\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.618684 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.618548    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-tmp-dir\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.618684 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.618617    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-model-cache\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.618684 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.618652    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-home\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.620520 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.620497    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-dshm\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.620766 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.620749    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39661479-8596-4419-a06f-6d498558ad20-tls-certs\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.627060 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.627036    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpnll\" (UniqueName: \"kubernetes.io/projected/39661479-8596-4419-a06f-6d498558ad20-kube-api-access-dpnll\") pod \"e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") " pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.735475 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.735395    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:34:56.861612 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:56.861587    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"]
Jul 29 18:34:56.864299 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:34:56.864265    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39661479_8596_4419_a06f_6d498558ad20.slice/crio-ae4cc2f7b69f3823d765006bd9937f3ad322f787a9bbe54e9719d2034d9fd8da WatchSource:0}: Error finding container ae4cc2f7b69f3823d765006bd9937f3ad322f787a9bbe54e9719d2034d9fd8da: Status 404 returned error can't find the container with id ae4cc2f7b69f3823d765006bd9937f3ad322f787a9bbe54e9719d2034d9fd8da
Jul 29 18:34:57.394816 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:57.394781    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp" event={"ID":"39661479-8596-4419-a06f-6d498558ad20","Type":"ContainerStarted","Data":"0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4"}
Jul 29 18:34:57.394997 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:57.394827    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp" event={"ID":"39661479-8596-4419-a06f-6d498558ad20","Type":"ContainerStarted","Data":"ae4cc2f7b69f3823d765006bd9937f3ad322f787a9bbe54e9719d2034d9fd8da"}
Jul 29 18:34:57.414541 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:34:57.414492    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp" podStartSLOduration=1.414476861 podStartE2EDuration="1.414476861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:57.412407087 +0000 UTC m=+2566.920790675" watchObservedRunningTime="2026-07-29 18:34:57.414476861 +0000 UTC m=+2566.922860449"
Jul 29 18:35:05.730212 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:05.730177    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:35:05.788762 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:05.788733    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:35:06.736095 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:06.736057    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:35:06.736095 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:06.736103    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:35:06.748838 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:06.748811    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:35:07.445689 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:07.445661    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:35:40.046920 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:40.046829    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:35:40.147075 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:40.147043    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:35:40.205174 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:40.205144    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:35:43.598435 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:43.598398    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:35:48.591211 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:48.591168    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"]
Jul 29 18:35:48.685295 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:35:48.685252    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:36:03.610068 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.610026    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp" podUID="39661479-8596-4419-a06f-6d498558ad20" containerName="main" containerID="cri-o://0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4" gracePeriod=45
Jul 29 18:36:03.855244 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.855218    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:36:03.955838 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.955810    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-model-cache\") pod \"39661479-8596-4419-a06f-6d498558ad20\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") "
Jul 29 18:36:03.956012 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.955850    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-dshm\") pod \"39661479-8596-4419-a06f-6d498558ad20\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") "
Jul 29 18:36:03.956012 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.955878    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39661479-8596-4419-a06f-6d498558ad20-tls-certs\") pod \"39661479-8596-4419-a06f-6d498558ad20\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") "
Jul 29 18:36:03.956012 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.955908    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-tmp-dir\") pod \"39661479-8596-4419-a06f-6d498558ad20\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") "
Jul 29 18:36:03.956012 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.955931    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-home\") pod \"39661479-8596-4419-a06f-6d498558ad20\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") "
Jul 29 18:36:03.956012 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.955959    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpnll\" (UniqueName: \"kubernetes.io/projected/39661479-8596-4419-a06f-6d498558ad20-kube-api-access-dpnll\") pod \"39661479-8596-4419-a06f-6d498558ad20\" (UID: \"39661479-8596-4419-a06f-6d498558ad20\") "
Jul 29 18:36:03.956266 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.956081    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-model-cache" (OuterVolumeSpecName: "model-cache") pod "39661479-8596-4419-a06f-6d498558ad20" (UID: "39661479-8596-4419-a06f-6d498558ad20"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:36:03.956266 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.956181    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-home" (OuterVolumeSpecName: "home") pod "39661479-8596-4419-a06f-6d498558ad20" (UID: "39661479-8596-4419-a06f-6d498558ad20"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:36:03.956266 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.956192    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "39661479-8596-4419-a06f-6d498558ad20" (UID: "39661479-8596-4419-a06f-6d498558ad20"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:36:03.958120 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.958085    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39661479-8596-4419-a06f-6d498558ad20-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "39661479-8596-4419-a06f-6d498558ad20" (UID: "39661479-8596-4419-a06f-6d498558ad20"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:36:03.958120 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.958110    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39661479-8596-4419-a06f-6d498558ad20-kube-api-access-dpnll" (OuterVolumeSpecName: "kube-api-access-dpnll") pod "39661479-8596-4419-a06f-6d498558ad20" (UID: "39661479-8596-4419-a06f-6d498558ad20"). InnerVolumeSpecName "kube-api-access-dpnll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:36:03.958259 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:03.958111    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-dshm" (OuterVolumeSpecName: "dshm") pod "39661479-8596-4419-a06f-6d498558ad20" (UID: "39661479-8596-4419-a06f-6d498558ad20"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:36:04.057028 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.056996    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-model-cache\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:36:04.057028 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.057022    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-dshm\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:36:04.057028 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.057031    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39661479-8596-4419-a06f-6d498558ad20-tls-certs\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:36:04.057234 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.057040    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-tmp-dir\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:36:04.057234 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.057047    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39661479-8596-4419-a06f-6d498558ad20-home\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:36:04.057234 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.057055    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpnll\" (UniqueName: \"kubernetes.io/projected/39661479-8596-4419-a06f-6d498558ad20-kube-api-access-dpnll\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:36:04.662872 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.662838    2556 generic.go:356] "Generic (PLEG): container finished" podID="39661479-8596-4419-a06f-6d498558ad20" containerID="0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4" exitCode=0
Jul 29 18:36:04.663441 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.662880    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"
Jul 29 18:36:04.663441 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.662906    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp" event={"ID":"39661479-8596-4419-a06f-6d498558ad20","Type":"ContainerDied","Data":"0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4"}
Jul 29 18:36:04.663441 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.662937    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp" event={"ID":"39661479-8596-4419-a06f-6d498558ad20","Type":"ContainerDied","Data":"ae4cc2f7b69f3823d765006bd9937f3ad322f787a9bbe54e9719d2034d9fd8da"}
Jul 29 18:36:04.663441 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.662953    2556 scope.go:117] "RemoveContainer" containerID="0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4"
Jul 29 18:36:04.671491 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.671472    2556 scope.go:117] "RemoveContainer" containerID="0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4"
Jul 29 18:36:04.671765 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:36:04.671746    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4\": container with ID starting with 0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4 not found: ID does not exist" containerID="0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4"
Jul 29 18:36:04.671804 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.671775    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4"} err="failed to get container status \"0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4\": rpc error: code = NotFound desc = could not find container \"0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4\": container with ID starting with 0155bb52992f56127b9071811c0db852c406f8f6ad3fd237ba01a80b64a6f1b4 not found: ID does not exist"
Jul 29 18:36:04.683179 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.683158    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"]
Jul 29 18:36:04.686930 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:04.686911    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-8be27fe2/e2e-lifecycle-model-8be27fe2-kserve-5ccbb78c8c-8dqbp"]
Jul 29 18:36:05.078050 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:05.077977    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39661479-8596-4419-a06f-6d498558ad20" path="/var/lib/kubelet/pods/39661479-8596-4419-a06f-6d498558ad20/volumes"
Jul 29 18:36:25.180691 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:25.180658    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:36:25.807754 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:25.807715    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:36:31.590358 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:31.590323    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:36:45.960461 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:45.960419    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:36:46.059005 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:46.058971    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:36:46.243703 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:46.243583    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:36:46.480525 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:46.480478    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:36:57.871642 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:57.871601    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:36:57.964174 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:57.964135    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:36:58.064594 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:36:58.064564    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:19.319189 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:19.319098    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:19.419969 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:19.419937    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:19.520415 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:19.520382    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:20.965947 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:20.965907    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:26.477248 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:26.477214    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:27.627952 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:27.627903    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:27.727117 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:27.727076    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:28.349845 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:28.349803    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:34.705436 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:34.705386    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:34.811506 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:34.811468    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:34.905722 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:34.905679    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:37:45.490019 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:37:45.489974    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:39:49.209505 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:39:49.209471    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:39:55.375557 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:39:55.375515    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:39:55.474043 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:39:55.474005    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:39:55.758223 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:39:55.758192    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:39:56.101088 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:39:56.101006    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:39:56.505268 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:39:56.505231    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:39:59.268007 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:39:59.267975    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:39:59.369121 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:39:59.369093    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:39:59.519896 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:39:59.519805    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:39:59.829824 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:39:59.829741    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:00.076960 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:00.076925    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:00.156827 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:00.156800    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:24.252611 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:24.252573    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:24.573459 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:24.573383    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:24.876684 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:24.876579    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:25.108323 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:25.108282    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:25.848277 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:25.848243    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:25.898751 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:25.898719    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:27.434001 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:27.433967    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:27.537091 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:27.537048    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:27.675238 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:27.675189    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:27.931395 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:27.931334    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:28.088300 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.088266    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:28.185885 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.185780    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:40:28.257445 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.257418    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"]
Jul 29 18:40:28.257707 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.257696    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39661479-8596-4419-a06f-6d498558ad20" containerName="main"
Jul 29 18:40:28.257751 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.257709    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="39661479-8596-4419-a06f-6d498558ad20" containerName="main"
Jul 29 18:40:28.257787 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.257768    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="39661479-8596-4419-a06f-6d498558ad20" containerName="main"
Jul 29 18:40:28.260560 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.260544    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:40:28.263003 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.262975    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-084953\""
Jul 29 18:40:28.263120 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.263022    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-084953-dockercfg-j85dq\""
Jul 29 18:40:28.270878 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.270853    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"]
Jul 29 18:40:28.356752 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.356718    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzs8v\" (UniqueName: \"kubernetes.io/projected/4a8f4832-918b-41b2-83b6-34af9157e9ab-kube-api-access-tzs8v\") pod \"payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc\" (UID: \"4a8f4832-918b-41b2-83b6-34af9157e9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:40:28.356923 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.356758    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8f4832-918b-41b2-83b6-34af9157e9ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc\" (UID: \"4a8f4832-918b-41b2-83b6-34af9157e9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:40:28.458108 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.458026    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzs8v\" (UniqueName: \"kubernetes.io/projected/4a8f4832-918b-41b2-83b6-34af9157e9ab-kube-api-access-tzs8v\") pod \"payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc\" (UID: \"4a8f4832-918b-41b2-83b6-34af9157e9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:40:28.458108 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.458064    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8f4832-918b-41b2-83b6-34af9157e9ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc\" (UID: \"4a8f4832-918b-41b2-83b6-34af9157e9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:40:28.458738 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.458719    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8f4832-918b-41b2-83b6-34af9157e9ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc\" (UID: \"4a8f4832-918b-41b2-83b6-34af9157e9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:40:28.465792 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.465768    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzs8v\" (UniqueName: \"kubernetes.io/projected/4a8f4832-918b-41b2-83b6-34af9157e9ab-kube-api-access-tzs8v\") pod \"payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc\" (UID: \"4a8f4832-918b-41b2-83b6-34af9157e9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:40:28.571447 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.571403    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:40:28.697918 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.697886    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"]
Jul 29 18:40:28.701754 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:40:28.701730    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8f4832_918b_41b2_83b6_34af9157e9ab.slice/crio-3990b93ce2de67bfb91a2565e3059450862198be68856457e8c03b61f2b03351 WatchSource:0}: Error finding container 3990b93ce2de67bfb91a2565e3059450862198be68856457e8c03b61f2b03351: Status 404 returned error can't find the container with id 3990b93ce2de67bfb91a2565e3059450862198be68856457e8c03b61f2b03351
Jul 29 18:40:28.704047 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.704026    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:40:28.708280 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:28.708217    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc" event={"ID":"4a8f4832-918b-41b2-83b6-34af9157e9ab","Type":"ContainerStarted","Data":"3990b93ce2de67bfb91a2565e3059450862198be68856457e8c03b61f2b03351"}
Jul 29 18:40:31.721909 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:31.721865    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc" event={"ID":"4a8f4832-918b-41b2-83b6-34af9157e9ab","Type":"ContainerStarted","Data":"f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8"}
Jul 29 18:40:31.722384 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:31.722006    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:40:31.736703 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:31.736604    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc" podStartSLOduration=0.958303446 podStartE2EDuration="3.736587529s" totalImagesPullingTime="2.778284083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:40:31.736524132 +0000 UTC m=+2901.244907717" watchObservedRunningTime="2026-07-29 18:40:31.736587529 +0000 UTC m=+2901.244971121"
Jul 29 18:40:42.728294 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:42.728265    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:40:52.419421 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:40:52.419384    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:41:03.920296 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:03.920260    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"]
Jul 29 18:41:03.924730 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:03.924713    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:03.928275 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:03.928255    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-54bd5a\"/\"default-dockercfg-5dpfn\""
Jul 29 18:41:03.928402 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:03.928328    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-54bd5a\"/\"test-model-54bd5a-kserve-self-signed-certs\""
Jul 29 18:41:03.928402 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:03.928256    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-54bd5a\"/\"openshift-service-ca.crt\""
Jul 29 18:41:03.928501 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:03.928379    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-54bd5a\"/\"kube-root-ca.crt\""
Jul 29 18:41:03.932747 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:03.932728    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"]
Jul 29 18:41:03.947223 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:03.947196    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:41:04.045251 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.045223    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-dshm\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.045409 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.045255    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-tmp-dir\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.045409 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.045336    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80a2e812-38d2-4506-bb93-2487fb531c9e-tls-certs\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.045409 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.045378    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-model-cache\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.045409 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.045402    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-home\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.045570 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.045435    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftmv2\" (UniqueName: \"kubernetes.io/projected/80a2e812-38d2-4506-bb93-2487fb531c9e-kube-api-access-ftmv2\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.146592 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.146563    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftmv2\" (UniqueName: \"kubernetes.io/projected/80a2e812-38d2-4506-bb93-2487fb531c9e-kube-api-access-ftmv2\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.146592 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.146595    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-dshm\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.146780 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.146618    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-tmp-dir\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.146780 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.146665    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80a2e812-38d2-4506-bb93-2487fb531c9e-tls-certs\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.146780 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.146761    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-model-cache\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.146898 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.146791    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-home\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.147131 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.147097    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-tmp-dir\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.147131 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.147126    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-home\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.147287 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.147157    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-model-cache\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.148916 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.148893    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-dshm\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.149145 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.149125    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80a2e812-38d2-4506-bb93-2487fb531c9e-tls-certs\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.154456 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.154438    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftmv2\" (UniqueName: \"kubernetes.io/projected/80a2e812-38d2-4506-bb93-2487fb531c9e-kube-api-access-ftmv2\") pod \"test-model-54bd5a-kserve-845596895f-v7c7r\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") " pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.236474 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.236395    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:04.363084 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.363060    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"]
Jul 29 18:41:04.365511 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:41:04.365484    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80a2e812_38d2_4506_bb93_2487fb531c9e.slice/crio-a26ce10a7ebf7e7ba108fbf8148cf3b52652540f92f97e70d4f75d63e9594102 WatchSource:0}: Error finding container a26ce10a7ebf7e7ba108fbf8148cf3b52652540f92f97e70d4f75d63e9594102: Status 404 returned error can't find the container with id a26ce10a7ebf7e7ba108fbf8148cf3b52652540f92f97e70d4f75d63e9594102
Jul 29 18:41:04.859090 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.859050    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r" event={"ID":"80a2e812-38d2-4506-bb93-2487fb531c9e","Type":"ContainerStarted","Data":"f90d071d716a81c82cd395c07a9339e4f7ee465dc4dedd94f376fef7686edb88"}
Jul 29 18:41:04.859090 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.859093    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r" event={"ID":"80a2e812-38d2-4506-bb93-2487fb531c9e","Type":"ContainerStarted","Data":"a26ce10a7ebf7e7ba108fbf8148cf3b52652540f92f97e70d4f75d63e9594102"}
Jul 29 18:41:04.875672 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:04.875625    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r" podStartSLOduration=1.8756073039999999 podStartE2EDuration="1.875607304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:41:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:41:04.875425549 +0000 UTC m=+2934.383809138" watchObservedRunningTime="2026-07-29 18:41:04.875607304 +0000 UTC m=+2934.383990892"
Jul 29 18:41:14.236669 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:14.236634    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:14.237061 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:14.236780    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:14.248965 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:14.248940    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:14.714504 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:14.714467    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:41:14.912226 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:14.912196    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:41:35.429898 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.429818    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"]
Jul 29 18:41:35.433033 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.433017    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.436034 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.436011    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:41:35.436590 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.436561    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-084953\"/\"test-model-084953-kserve-self-signed-certs\""
Jul 29 18:41:35.436699 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.436576    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-084953\"/\"kube-root-ca.crt\""
Jul 29 18:41:35.436699 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.436640    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-084953\"/\"default-dockercfg-xktnc\""
Jul 29 18:41:35.436699 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.436690    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-084953\"/\"openshift-service-ca.crt\""
Jul 29 18:41:35.441236 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.441209    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"]
Jul 29 18:41:35.502439 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.502402    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-dshm\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.502613 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.502458    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-home\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.502613 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.502482    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxsw8\" (UniqueName: \"kubernetes.io/projected/6f793595-eb17-4275-8b8f-371e5870914a-kube-api-access-fxsw8\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.502613 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.502527    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-model-cache\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.502613 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.502544    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f793595-eb17-4275-8b8f-371e5870914a-tls-certs\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.502757 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.502615    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-tmp-dir\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.603783 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.603750    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-model-cache\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.603783 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.603786    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f793595-eb17-4275-8b8f-371e5870914a-tls-certs\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.604026 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.603811    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-tmp-dir\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.604026 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.603833    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-dshm\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.604026 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.603882    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-home\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.604026 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.603917    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsw8\" (UniqueName: \"kubernetes.io/projected/6f793595-eb17-4275-8b8f-371e5870914a-kube-api-access-fxsw8\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.604280 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.604255    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-model-cache\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.604387 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.604296    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-home\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.604387 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.604326    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-tmp-dir\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.606148 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.606126    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-dshm\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.606431 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.606415    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f793595-eb17-4275-8b8f-371e5870914a-tls-certs\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.611958 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.611933    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxsw8\" (UniqueName: \"kubernetes.io/projected/6f793595-eb17-4275-8b8f-371e5870914a-kube-api-access-fxsw8\") pod \"test-model-084953-kserve-757f76cff-g46dg\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") " pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.743585 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.743497    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:35.870406 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.870350    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"]
Jul 29 18:41:35.873651 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:41:35.873620    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f793595_eb17_4275_8b8f_371e5870914a.slice/crio-680b6d6ad2ead8ee76a1eb18d5a73623c87028986ba7e32445d29e46c20eeed8 WatchSource:0}: Error finding container 680b6d6ad2ead8ee76a1eb18d5a73623c87028986ba7e32445d29e46c20eeed8: Status 404 returned error can't find the container with id 680b6d6ad2ead8ee76a1eb18d5a73623c87028986ba7e32445d29e46c20eeed8
Jul 29 18:41:35.988730 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.988692    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg" event={"ID":"6f793595-eb17-4275-8b8f-371e5870914a","Type":"ContainerStarted","Data":"80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf"}
Jul 29 18:41:35.988899 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:35.988740    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg" event={"ID":"6f793595-eb17-4275-8b8f-371e5870914a","Type":"ContainerStarted","Data":"680b6d6ad2ead8ee76a1eb18d5a73623c87028986ba7e32445d29e46c20eeed8"}
Jul 29 18:41:36.004105 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:36.004017    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg" podStartSLOduration=1.004002709 podStartE2EDuration="1.004002709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:41:36.003556786 +0000 UTC m=+2965.511940377" watchObservedRunningTime="2026-07-29 18:41:36.004002709 +0000 UTC m=+2965.512386300"
Jul 29 18:41:45.744692 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:45.744655    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:45.745152 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:45.744750    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:45.758565 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:45.758536    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:41:46.045086 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:41:46.045008    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:42:16.992642 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:16.992604    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"]
Jul 29 18:42:17.188559 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:17.188519    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:17.886643 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:17.886610    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"]
Jul 29 18:42:17.959345 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:17.959313    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:28.625206 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:28.625164    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:28.725081 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:28.725051    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:28.802818 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:28.802783    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:28.854967 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:28.854940    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:29.169624 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:29.169592    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:32.011922 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.011885    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r" podUID="80a2e812-38d2-4506-bb93-2487fb531c9e" containerName="main" containerID="cri-o://f90d071d716a81c82cd395c07a9339e4f7ee465dc4dedd94f376fef7686edb88" gracePeriod=45
Jul 29 18:42:32.221399 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.221344    2556 generic.go:356] "Generic (PLEG): container finished" podID="80a2e812-38d2-4506-bb93-2487fb531c9e" containerID="f90d071d716a81c82cd395c07a9339e4f7ee465dc4dedd94f376fef7686edb88" exitCode=0
Jul 29 18:42:32.221559 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.221426    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r" event={"ID":"80a2e812-38d2-4506-bb93-2487fb531c9e","Type":"ContainerDied","Data":"f90d071d716a81c82cd395c07a9339e4f7ee465dc4dedd94f376fef7686edb88"}
Jul 29 18:42:32.257059 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.257034    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:42:32.279255 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.279189    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-dshm\") pod \"80a2e812-38d2-4506-bb93-2487fb531c9e\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") "
Jul 29 18:42:32.279255 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.279238    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80a2e812-38d2-4506-bb93-2487fb531c9e-tls-certs\") pod \"80a2e812-38d2-4506-bb93-2487fb531c9e\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") "
Jul 29 18:42:32.279255 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.279255    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-model-cache\") pod \"80a2e812-38d2-4506-bb93-2487fb531c9e\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") "
Jul 29 18:42:32.279521 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.279284    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftmv2\" (UniqueName: \"kubernetes.io/projected/80a2e812-38d2-4506-bb93-2487fb531c9e-kube-api-access-ftmv2\") pod \"80a2e812-38d2-4506-bb93-2487fb531c9e\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") "
Jul 29 18:42:32.279521 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.279327    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-home\") pod \"80a2e812-38d2-4506-bb93-2487fb531c9e\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") "
Jul 29 18:42:32.279521 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.279394    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-tmp-dir\") pod \"80a2e812-38d2-4506-bb93-2487fb531c9e\" (UID: \"80a2e812-38d2-4506-bb93-2487fb531c9e\") "
Jul 29 18:42:32.279680 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.279552    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-model-cache" (OuterVolumeSpecName: "model-cache") pod "80a2e812-38d2-4506-bb93-2487fb531c9e" (UID: "80a2e812-38d2-4506-bb93-2487fb531c9e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:32.279680 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.279666    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-model-cache\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:32.279785 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.279710    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-home" (OuterVolumeSpecName: "home") pod "80a2e812-38d2-4506-bb93-2487fb531c9e" (UID: "80a2e812-38d2-4506-bb93-2487fb531c9e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:32.279785 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.279759    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "80a2e812-38d2-4506-bb93-2487fb531c9e" (UID: "80a2e812-38d2-4506-bb93-2487fb531c9e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:32.281583 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.281561    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80a2e812-38d2-4506-bb93-2487fb531c9e-kube-api-access-ftmv2" (OuterVolumeSpecName: "kube-api-access-ftmv2") pod "80a2e812-38d2-4506-bb93-2487fb531c9e" (UID: "80a2e812-38d2-4506-bb93-2487fb531c9e"). InnerVolumeSpecName "kube-api-access-ftmv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:32.281700 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.281591    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-dshm" (OuterVolumeSpecName: "dshm") pod "80a2e812-38d2-4506-bb93-2487fb531c9e" (UID: "80a2e812-38d2-4506-bb93-2487fb531c9e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:32.281760 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.281724    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80a2e812-38d2-4506-bb93-2487fb531c9e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "80a2e812-38d2-4506-bb93-2487fb531c9e" (UID: "80a2e812-38d2-4506-bb93-2487fb531c9e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:42:32.380377 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.380341    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-home\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:32.380547 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.380408    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-tmp-dir\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:32.380547 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.380424    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80a2e812-38d2-4506-bb93-2487fb531c9e-dshm\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:32.380547 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.380432    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80a2e812-38d2-4506-bb93-2487fb531c9e-tls-certs\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:32.380547 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.380440    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftmv2\" (UniqueName: \"kubernetes.io/projected/80a2e812-38d2-4506-bb93-2487fb531c9e-kube-api-access-ftmv2\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:32.906229 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:32.906186    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg" podUID="6f793595-eb17-4275-8b8f-371e5870914a" containerName="main" containerID="cri-o://80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf" gracePeriod=45
Jul 29 18:42:33.146999 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.146970    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:42:33.187895 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.187815    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-tmp-dir\") pod \"6f793595-eb17-4275-8b8f-371e5870914a\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") "
Jul 29 18:42:33.187895 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.187861    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-dshm\") pod \"6f793595-eb17-4275-8b8f-371e5870914a\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") "
Jul 29 18:42:33.187895 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.187882    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-home\") pod \"6f793595-eb17-4275-8b8f-371e5870914a\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") "
Jul 29 18:42:33.188136 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.187910    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-model-cache\") pod \"6f793595-eb17-4275-8b8f-371e5870914a\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") "
Jul 29 18:42:33.188136 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.187944    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxsw8\" (UniqueName: \"kubernetes.io/projected/6f793595-eb17-4275-8b8f-371e5870914a-kube-api-access-fxsw8\") pod \"6f793595-eb17-4275-8b8f-371e5870914a\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") "
Jul 29 18:42:33.188136 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.187970    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f793595-eb17-4275-8b8f-371e5870914a-tls-certs\") pod \"6f793595-eb17-4275-8b8f-371e5870914a\" (UID: \"6f793595-eb17-4275-8b8f-371e5870914a\") "
Jul 29 18:42:33.188293 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.188147    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f793595-eb17-4275-8b8f-371e5870914a" (UID: "6f793595-eb17-4275-8b8f-371e5870914a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:33.188293 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.188156    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-home" (OuterVolumeSpecName: "home") pod "6f793595-eb17-4275-8b8f-371e5870914a" (UID: "6f793595-eb17-4275-8b8f-371e5870914a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:33.188293 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.188252    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f793595-eb17-4275-8b8f-371e5870914a" (UID: "6f793595-eb17-4275-8b8f-371e5870914a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:33.190279 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.190252    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f793595-eb17-4275-8b8f-371e5870914a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f793595-eb17-4275-8b8f-371e5870914a" (UID: "6f793595-eb17-4275-8b8f-371e5870914a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:42:33.190372 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.190269    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-dshm" (OuterVolumeSpecName: "dshm") pod "6f793595-eb17-4275-8b8f-371e5870914a" (UID: "6f793595-eb17-4275-8b8f-371e5870914a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:33.190372 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.190293    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f793595-eb17-4275-8b8f-371e5870914a-kube-api-access-fxsw8" (OuterVolumeSpecName: "kube-api-access-fxsw8") pod "6f793595-eb17-4275-8b8f-371e5870914a" (UID: "6f793595-eb17-4275-8b8f-371e5870914a"). InnerVolumeSpecName "kube-api-access-fxsw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:33.227317 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.227290    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"
Jul 29 18:42:33.227317 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.227298    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r" event={"ID":"80a2e812-38d2-4506-bb93-2487fb531c9e","Type":"ContainerDied","Data":"a26ce10a7ebf7e7ba108fbf8148cf3b52652540f92f97e70d4f75d63e9594102"}
Jul 29 18:42:33.227520 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.227343    2556 scope.go:117] "RemoveContainer" containerID="f90d071d716a81c82cd395c07a9339e4f7ee465dc4dedd94f376fef7686edb88"
Jul 29 18:42:33.228699 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.228674    2556 generic.go:356] "Generic (PLEG): container finished" podID="6f793595-eb17-4275-8b8f-371e5870914a" containerID="80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf" exitCode=0
Jul 29 18:42:33.228816 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.228721    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"
Jul 29 18:42:33.228816 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.228720    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg" event={"ID":"6f793595-eb17-4275-8b8f-371e5870914a","Type":"ContainerDied","Data":"80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf"}
Jul 29 18:42:33.228937 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.228837    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg" event={"ID":"6f793595-eb17-4275-8b8f-371e5870914a","Type":"ContainerDied","Data":"680b6d6ad2ead8ee76a1eb18d5a73623c87028986ba7e32445d29e46c20eeed8"}
Jul 29 18:42:33.236730 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.236714    2556 scope.go:117] "RemoveContainer" containerID="80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf"
Jul 29 18:42:33.243326 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.243300    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"]
Jul 29 18:42:33.245748 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.245728    2556 scope.go:117] "RemoveContainer" containerID="80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf"
Jul 29 18:42:33.246035 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.246014    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-54bd5a/test-model-54bd5a-kserve-845596895f-v7c7r"]
Jul 29 18:42:33.246035 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:42:33.246020    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf\": container with ID starting with 80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf not found: ID does not exist" containerID="80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf"
Jul 29 18:42:33.246166 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.246049    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf"} err="failed to get container status \"80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf\": rpc error: code = NotFound desc = could not find container \"80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf\": container with ID starting with 80a9ebf4f9cfd6399b712f1216542a78cf1c5488a1f2a9bd9342e80e55540fdf not found: ID does not exist"
Jul 29 18:42:33.257915 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.257887    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"]
Jul 29 18:42:33.261493 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.261467    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-084953/test-model-084953-kserve-757f76cff-g46dg"]
Jul 29 18:42:33.265717 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:42:33.265691    2556 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f793595_eb17_4275_8b8f_371e5870914a.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f793595_eb17_4275_8b8f_371e5870914a.slice/crio-680b6d6ad2ead8ee76a1eb18d5a73623c87028986ba7e32445d29e46c20eeed8\": RecentStats: unable to find data in memory cache]"
Jul 29 18:42:33.289086 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.289055    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f793595-eb17-4275-8b8f-371e5870914a-tls-certs\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:33.289086 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.289080    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-tmp-dir\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:33.289086 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.289089    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-dshm\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:33.289286 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.289096    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-home\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:33.289286 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.289104    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f793595-eb17-4275-8b8f-371e5870914a-model-cache\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:33.289286 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:33.289113    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxsw8\" (UniqueName: \"kubernetes.io/projected/6f793595-eb17-4275-8b8f-371e5870914a-kube-api-access-fxsw8\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:34.997723 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:34.997692    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:35.078388 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:35.078332    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f793595-eb17-4275-8b8f-371e5870914a" path="/var/lib/kubelet/pods/6f793595-eb17-4275-8b8f-371e5870914a/volumes"
Jul 29 18:42:35.078732 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:35.078718    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80a2e812-38d2-4506-bb93-2487fb531c9e" path="/var/lib/kubelet/pods/80a2e812-38d2-4506-bb93-2487fb531c9e/volumes"
Jul 29 18:42:35.097605 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:35.097574    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:35.282721 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:35.282640    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:35.526781 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:35.526749    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:40.682337 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:40.682297    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:46.414965 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.414926    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:46.440486 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.440441    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"]
Jul 29 18:42:46.440775 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.440734    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc" podUID="4a8f4832-918b-41b2-83b6-34af9157e9ab" containerName="payload-pre-processing" containerID="cri-o://f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8" gracePeriod=30
Jul 29 18:42:46.514756 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.514725    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:46.576619 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.576590    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:46.680607 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.680579    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:42:46.803296 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.803258    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzs8v\" (UniqueName: \"kubernetes.io/projected/4a8f4832-918b-41b2-83b6-34af9157e9ab-kube-api-access-tzs8v\") pod \"4a8f4832-918b-41b2-83b6-34af9157e9ab\" (UID: \"4a8f4832-918b-41b2-83b6-34af9157e9ab\") "
Jul 29 18:42:46.803296 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.803302    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8f4832-918b-41b2-83b6-34af9157e9ab-plugins-config-volume\") pod \"4a8f4832-918b-41b2-83b6-34af9157e9ab\" (UID: \"4a8f4832-918b-41b2-83b6-34af9157e9ab\") "
Jul 29 18:42:46.803689 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.803665    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a8f4832-918b-41b2-83b6-34af9157e9ab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a8f4832-918b-41b2-83b6-34af9157e9ab" (UID: "4a8f4832-918b-41b2-83b6-34af9157e9ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:42:46.805330 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.805307    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a8f4832-918b-41b2-83b6-34af9157e9ab-kube-api-access-tzs8v" (OuterVolumeSpecName: "kube-api-access-tzs8v") pod "4a8f4832-918b-41b2-83b6-34af9157e9ab" (UID: "4a8f4832-918b-41b2-83b6-34af9157e9ab"). InnerVolumeSpecName "kube-api-access-tzs8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:46.904441 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.904405    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzs8v\" (UniqueName: \"kubernetes.io/projected/4a8f4832-918b-41b2-83b6-34af9157e9ab-kube-api-access-tzs8v\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:46.904441 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.904440    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a8f4832-918b-41b2-83b6-34af9157e9ab-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:42:46.934769 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:46.934689    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:47.289410 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:47.289310    2556 generic.go:356] "Generic (PLEG): container finished" podID="4a8f4832-918b-41b2-83b6-34af9157e9ab" containerID="f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8" exitCode=0
Jul 29 18:42:47.289410 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:47.289358    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"
Jul 29 18:42:47.289410 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:47.289396    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc" event={"ID":"4a8f4832-918b-41b2-83b6-34af9157e9ab","Type":"ContainerDied","Data":"f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8"}
Jul 29 18:42:47.289775 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:47.289453    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc" event={"ID":"4a8f4832-918b-41b2-83b6-34af9157e9ab","Type":"ContainerDied","Data":"3990b93ce2de67bfb91a2565e3059450862198be68856457e8c03b61f2b03351"}
Jul 29 18:42:47.289775 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:47.289475    2556 scope.go:117] "RemoveContainer" containerID="f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8"
Jul 29 18:42:47.299242 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:47.299220    2556 scope.go:117] "RemoveContainer" containerID="f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8"
Jul 29 18:42:47.299532 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:42:47.299512    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8\": container with ID starting with f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8 not found: ID does not exist" containerID="f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8"
Jul 29 18:42:47.299610 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:47.299541    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8"} err="failed to get container status \"f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8\": rpc error: code = NotFound desc = could not find container \"f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8\": container with ID starting with f45d4a4cc372bf938ac7eaae7bb26e8866369d18e5ec8cae09a09a3e001705a8 not found: ID does not exist"
Jul 29 18:42:47.311301 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:47.311276    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"]
Jul 29 18:42:47.315310 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:47.315285    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-084953-794dd8959b-txwvc"]
Jul 29 18:42:49.078753 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:49.078721    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a8f4832-918b-41b2-83b6-34af9157e9ab" path="/var/lib/kubelet/pods/4a8f4832-918b-41b2-83b6-34af9157e9ab/volumes"
Jul 29 18:42:52.707756 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:52.707728    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:52.815415 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:52.815348    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:52.923395 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:52.923346    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:53.162148 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:53.162114    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:53.514529 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:53.514444    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:53.818878 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:53.818792    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:54.296994 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:54.296964    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:57.189997 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:57.189955    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:57.285901 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:57.285867    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:57.436487 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:57.436458    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:57.566622 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:57.566541    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:57.895042 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:57.895010    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:57.997263 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:57.997234    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:42:58.005057 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.005030    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"]
Jul 29 18:42:58.005336 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.005325    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a2e812-38d2-4506-bb93-2487fb531c9e" containerName="main"
Jul 29 18:42:58.005557 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.005338    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a2e812-38d2-4506-bb93-2487fb531c9e" containerName="main"
Jul 29 18:42:58.005557 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.005354    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a8f4832-918b-41b2-83b6-34af9157e9ab" containerName="payload-pre-processing"
Jul 29 18:42:58.005557 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.005386    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a8f4832-918b-41b2-83b6-34af9157e9ab" containerName="payload-pre-processing"
Jul 29 18:42:58.005557 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.005408    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f793595-eb17-4275-8b8f-371e5870914a" containerName="main"
Jul 29 18:42:58.005557 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.005415    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f793595-eb17-4275-8b8f-371e5870914a" containerName="main"
Jul 29 18:42:58.005557 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.005479    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a8f4832-918b-41b2-83b6-34af9157e9ab" containerName="payload-pre-processing"
Jul 29 18:42:58.005557 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.005487    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f793595-eb17-4275-8b8f-371e5870914a" containerName="main"
Jul 29 18:42:58.005557 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.005496    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="80a2e812-38d2-4506-bb93-2487fb531c9e" containerName="main"
Jul 29 18:42:58.008197 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.008181    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:42:58.010835 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.010813    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-109296\""
Jul 29 18:42:58.010953 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.010864    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-109296-dockercfg-fw7xd\""
Jul 29 18:42:58.025021 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.024995    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"]
Jul 29 18:42:58.037250 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.037223    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"]
Jul 29 18:42:58.040788 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.040773    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:42:58.048065 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.048040    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"]
Jul 29 18:42:58.090714 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.090681    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea23ce7-0077-4e03-8a53-76b38b05c13d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb\" (UID: \"8ea23ce7-0077-4e03-8a53-76b38b05c13d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:42:58.090878 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.090751    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv6q4\" (UniqueName: \"kubernetes.io/projected/8ea23ce7-0077-4e03-8a53-76b38b05c13d-kube-api-access-zv6q4\") pod \"payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb\" (UID: \"8ea23ce7-0077-4e03-8a53-76b38b05c13d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:42:58.191268 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.191165    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv6q4\" (UniqueName: \"kubernetes.io/projected/8ea23ce7-0077-4e03-8a53-76b38b05c13d-kube-api-access-zv6q4\") pod \"payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb\" (UID: \"8ea23ce7-0077-4e03-8a53-76b38b05c13d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:42:58.191268 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.191219    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8946d9-1328-4dd2-9190-539162c4b377-plugins-config-volume\") pod \"payload-processing-e2e-api-a-109296-5c4b486f46-5tv49\" (UID: \"bb8946d9-1328-4dd2-9190-539162c4b377\") " pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:42:58.191820 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.191267    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d22gp\" (UniqueName: \"kubernetes.io/projected/bb8946d9-1328-4dd2-9190-539162c4b377-kube-api-access-d22gp\") pod \"payload-processing-e2e-api-a-109296-5c4b486f46-5tv49\" (UID: \"bb8946d9-1328-4dd2-9190-539162c4b377\") " pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:42:58.191820 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.191310    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea23ce7-0077-4e03-8a53-76b38b05c13d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb\" (UID: \"8ea23ce7-0077-4e03-8a53-76b38b05c13d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:42:58.192124 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.192100    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea23ce7-0077-4e03-8a53-76b38b05c13d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb\" (UID: \"8ea23ce7-0077-4e03-8a53-76b38b05c13d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:42:58.200992 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.200962    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv6q4\" (UniqueName: \"kubernetes.io/projected/8ea23ce7-0077-4e03-8a53-76b38b05c13d-kube-api-access-zv6q4\") pod \"payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb\" (UID: \"8ea23ce7-0077-4e03-8a53-76b38b05c13d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:42:58.292433 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.292398    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8946d9-1328-4dd2-9190-539162c4b377-plugins-config-volume\") pod \"payload-processing-e2e-api-a-109296-5c4b486f46-5tv49\" (UID: \"bb8946d9-1328-4dd2-9190-539162c4b377\") " pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:42:58.292623 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.292455    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d22gp\" (UniqueName: \"kubernetes.io/projected/bb8946d9-1328-4dd2-9190-539162c4b377-kube-api-access-d22gp\") pod \"payload-processing-e2e-api-a-109296-5c4b486f46-5tv49\" (UID: \"bb8946d9-1328-4dd2-9190-539162c4b377\") " pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:42:58.293137 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.293115    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8946d9-1328-4dd2-9190-539162c4b377-plugins-config-volume\") pod \"payload-processing-e2e-api-a-109296-5c4b486f46-5tv49\" (UID: \"bb8946d9-1328-4dd2-9190-539162c4b377\") " pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:42:58.300210 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.300179    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d22gp\" (UniqueName: \"kubernetes.io/projected/bb8946d9-1328-4dd2-9190-539162c4b377-kube-api-access-d22gp\") pod \"payload-processing-e2e-api-a-109296-5c4b486f46-5tv49\" (UID: \"bb8946d9-1328-4dd2-9190-539162c4b377\") " pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:42:58.318094 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.318062    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:42:58.351189 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.351156    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:42:58.465206 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.465083    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"]
Jul 29 18:42:58.468750 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:42:58.468710    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea23ce7_0077_4e03_8a53_76b38b05c13d.slice/crio-8493babc7d0551fc90416886600c3361ce13b5f6e0d771e5c84aff688e5e7394 WatchSource:0}: Error finding container 8493babc7d0551fc90416886600c3361ce13b5f6e0d771e5c84aff688e5e7394: Status 404 returned error can't find the container with id 8493babc7d0551fc90416886600c3361ce13b5f6e0d771e5c84aff688e5e7394
Jul 29 18:42:58.488006 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:58.487985    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"]
Jul 29 18:42:58.498413 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:42:58.498382    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb8946d9_1328_4dd2_9190_539162c4b377.slice/crio-288bbeef72397596251691667fa8e252895b1772d070a0b7f65ab8442d48b0d5 WatchSource:0}: Error finding container 288bbeef72397596251691667fa8e252895b1772d070a0b7f65ab8442d48b0d5: Status 404 returned error can't find the container with id 288bbeef72397596251691667fa8e252895b1772d070a0b7f65ab8442d48b0d5
Jul 29 18:42:59.341837 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:59.341805    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49" event={"ID":"bb8946d9-1328-4dd2-9190-539162c4b377","Type":"ContainerStarted","Data":"61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6"}
Jul 29 18:42:59.342305 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:59.341854    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49" event={"ID":"bb8946d9-1328-4dd2-9190-539162c4b377","Type":"ContainerStarted","Data":"288bbeef72397596251691667fa8e252895b1772d070a0b7f65ab8442d48b0d5"}
Jul 29 18:42:59.342305 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:59.341894    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:42:59.343299 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:59.343276    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb" event={"ID":"8ea23ce7-0077-4e03-8a53-76b38b05c13d","Type":"ContainerStarted","Data":"6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707"}
Jul 29 18:42:59.343445 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:59.343312    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb" event={"ID":"8ea23ce7-0077-4e03-8a53-76b38b05c13d","Type":"ContainerStarted","Data":"8493babc7d0551fc90416886600c3361ce13b5f6e0d771e5c84aff688e5e7394"}
Jul 29 18:42:59.343445 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:59.343346    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:42:59.357240 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:59.357195    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49" podStartSLOduration=1.357180306 podStartE2EDuration="1.357180306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:42:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:42:59.356489695 +0000 UTC m=+3048.864873288" watchObservedRunningTime="2026-07-29 18:42:59.357180306 +0000 UTC m=+3048.865563893"
Jul 29 18:42:59.372260 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:42:59.372212    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb" podStartSLOduration=2.372194296 podStartE2EDuration="2.372194296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:42:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:42:59.371398926 +0000 UTC m=+3048.879782514" watchObservedRunningTime="2026-07-29 18:42:59.372194296 +0000 UTC m=+3048.880577886"
Jul 29 18:43:10.350012 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:10.349978    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:43:10.350495 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:10.350050    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:43:22.183779 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:22.183723    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:43:22.661433 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:22.661397    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:43:22.970032 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:22.969946    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:43:23.069702 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:23.069673    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:43:26.334560 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:26.334525    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:43:26.429520 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:26.429486    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:43:26.511896 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:26.511865    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:43:26.761479 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:26.761444    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:43:27.100906 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.100827    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:43:27.199811 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.199776    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:43:27.231920 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.231891    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"]
Jul 29 18:43:27.235060 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.235044    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:43:27.237613 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.237594    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-95c5f7\""
Jul 29 18:43:27.247413 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.247393    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"]
Jul 29 18:43:27.300086 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.300055    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"]
Jul 29 18:43:27.303207 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.303191    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:43:27.305611 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.305589    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-95c5f7-dockercfg-x8wks\""
Jul 29 18:43:27.312661 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.312638    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"]
Jul 29 18:43:27.325299 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.325275    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8c1ea99-6c41-47fe-9705-93832131ed1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8\" (UID: \"a8c1ea99-6c41-47fe-9705-93832131ed1b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:43:27.325433 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.325322    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8\" (UID: \"02edcbe6-2b55-4296-9fb3-7d34a62d68b1\") " pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:43:27.325433 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.325376    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvg58\" (UniqueName: \"kubernetes.io/projected/a8c1ea99-6c41-47fe-9705-93832131ed1b-kube-api-access-mvg58\") pod \"payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8\" (UID: \"a8c1ea99-6c41-47fe-9705-93832131ed1b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:43:27.325433 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.325423    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxnz8\" (UniqueName: \"kubernetes.io/projected/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-kube-api-access-dxnz8\") pod \"payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8\" (UID: \"02edcbe6-2b55-4296-9fb3-7d34a62d68b1\") " pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:43:27.425878 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.425845    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8\" (UID: \"02edcbe6-2b55-4296-9fb3-7d34a62d68b1\") " pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:43:27.426328 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.425883    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvg58\" (UniqueName: \"kubernetes.io/projected/a8c1ea99-6c41-47fe-9705-93832131ed1b-kube-api-access-mvg58\") pod \"payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8\" (UID: \"a8c1ea99-6c41-47fe-9705-93832131ed1b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:43:27.426328 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.426014    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxnz8\" (UniqueName: \"kubernetes.io/projected/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-kube-api-access-dxnz8\") pod \"payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8\" (UID: \"02edcbe6-2b55-4296-9fb3-7d34a62d68b1\") " pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:43:27.426328 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.426080    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8c1ea99-6c41-47fe-9705-93832131ed1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8\" (UID: \"a8c1ea99-6c41-47fe-9705-93832131ed1b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:43:27.426755 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.426733    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8\" (UID: \"02edcbe6-2b55-4296-9fb3-7d34a62d68b1\") " pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:43:27.426825 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.426799    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8c1ea99-6c41-47fe-9705-93832131ed1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8\" (UID: \"a8c1ea99-6c41-47fe-9705-93832131ed1b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:43:27.437843 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.437822    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxnz8\" (UniqueName: \"kubernetes.io/projected/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-kube-api-access-dxnz8\") pod \"payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8\" (UID: \"02edcbe6-2b55-4296-9fb3-7d34a62d68b1\") " pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:43:27.438070 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.438050    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvg58\" (UniqueName: \"kubernetes.io/projected/a8c1ea99-6c41-47fe-9705-93832131ed1b-kube-api-access-mvg58\") pod \"payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8\" (UID: \"a8c1ea99-6c41-47fe-9705-93832131ed1b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:43:27.544212 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.544179    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:43:27.613464 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.613422    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:43:27.673314 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.673278    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"]
Jul 29 18:43:27.772688 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:27.772659    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"]
Jul 29 18:43:27.773920 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:43:27.773891    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c1ea99_6c41_47fe_9705_93832131ed1b.slice/crio-d3ed054d48b7e06311bd26de105598eaf90b56419ab7d711f4ec31b69136e313 WatchSource:0}: Error finding container d3ed054d48b7e06311bd26de105598eaf90b56419ab7d711f4ec31b69136e313: Status 404 returned error can't find the container with id d3ed054d48b7e06311bd26de105598eaf90b56419ab7d711f4ec31b69136e313
Jul 29 18:43:28.464414 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:28.464379    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8" event={"ID":"a8c1ea99-6c41-47fe-9705-93832131ed1b","Type":"ContainerStarted","Data":"c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02"}
Jul 29 18:43:28.464839 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:28.464439    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8" event={"ID":"a8c1ea99-6c41-47fe-9705-93832131ed1b","Type":"ContainerStarted","Data":"d3ed054d48b7e06311bd26de105598eaf90b56419ab7d711f4ec31b69136e313"}
Jul 29 18:43:28.464839 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:28.464482    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:43:28.466022 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:28.466002    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8" event={"ID":"02edcbe6-2b55-4296-9fb3-7d34a62d68b1","Type":"ContainerStarted","Data":"fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157"}
Jul 29 18:43:28.466102 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:28.466037    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8" event={"ID":"02edcbe6-2b55-4296-9fb3-7d34a62d68b1","Type":"ContainerStarted","Data":"9be739b8262387d16b1ae0797e1ed9bf3c33ddca41cf5421c997ef1c0ea4c066"}
Jul 29 18:43:28.466102 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:28.466055    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:43:28.481604 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:28.481560    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8" podStartSLOduration=1.481547881 podStartE2EDuration="1.481547881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:43:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:43:28.480590504 +0000 UTC m=+3077.988974091" watchObservedRunningTime="2026-07-29 18:43:28.481547881 +0000 UTC m=+3077.989931469"
Jul 29 18:43:28.496667 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:28.496622    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8" podStartSLOduration=1.496609056 podStartE2EDuration="1.496609056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:43:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:43:28.495995923 +0000 UTC m=+3078.004379513" watchObservedRunningTime="2026-07-29 18:43:28.496609056 +0000 UTC m=+3078.004992657"
Jul 29 18:43:39.472565 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:39.472527    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:43:39.473071 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:39.472626    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:43:51.336677 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:43:51.336638    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:09.632681 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:09.632643    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:09.657276 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:09.657245    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"]
Jul 29 18:44:09.657669 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:09.657496    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49" podUID="bb8946d9-1328-4dd2-9190-539162c4b377" containerName="payload-processing" containerID="cri-o://61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6" gracePeriod=30
Jul 29 18:44:09.667228 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:09.667201    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"]
Jul 29 18:44:09.667583 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:09.667558    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb" podUID="8ea23ce7-0077-4e03-8a53-76b38b05c13d" containerName="payload-pre-processing" containerID="cri-o://6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707" gracePeriod=30
Jul 29 18:44:09.698919 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:09.698892    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:09.755998 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:09.755968    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:09.812184 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:09.812149    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:09.918264 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:09.918242    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:44:09.957387 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:09.945951    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:44:10.102571 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.102536    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea23ce7-0077-4e03-8a53-76b38b05c13d-plugins-config-volume\") pod \"8ea23ce7-0077-4e03-8a53-76b38b05c13d\" (UID: \"8ea23ce7-0077-4e03-8a53-76b38b05c13d\") "
Jul 29 18:44:10.102747 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.102613    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d22gp\" (UniqueName: \"kubernetes.io/projected/bb8946d9-1328-4dd2-9190-539162c4b377-kube-api-access-d22gp\") pod \"bb8946d9-1328-4dd2-9190-539162c4b377\" (UID: \"bb8946d9-1328-4dd2-9190-539162c4b377\") "
Jul 29 18:44:10.102747 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.102646    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv6q4\" (UniqueName: \"kubernetes.io/projected/8ea23ce7-0077-4e03-8a53-76b38b05c13d-kube-api-access-zv6q4\") pod \"8ea23ce7-0077-4e03-8a53-76b38b05c13d\" (UID: \"8ea23ce7-0077-4e03-8a53-76b38b05c13d\") "
Jul 29 18:44:10.102747 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.102674    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8946d9-1328-4dd2-9190-539162c4b377-plugins-config-volume\") pod \"bb8946d9-1328-4dd2-9190-539162c4b377\" (UID: \"bb8946d9-1328-4dd2-9190-539162c4b377\") "
Jul 29 18:44:10.102912 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.102889    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ea23ce7-0077-4e03-8a53-76b38b05c13d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ea23ce7-0077-4e03-8a53-76b38b05c13d" (UID: "8ea23ce7-0077-4e03-8a53-76b38b05c13d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:44:10.103123 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.103096    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb8946d9-1328-4dd2-9190-539162c4b377-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb8946d9-1328-4dd2-9190-539162c4b377" (UID: "bb8946d9-1328-4dd2-9190-539162c4b377"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:44:10.104711 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.104692    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ea23ce7-0077-4e03-8a53-76b38b05c13d-kube-api-access-zv6q4" (OuterVolumeSpecName: "kube-api-access-zv6q4") pod "8ea23ce7-0077-4e03-8a53-76b38b05c13d" (UID: "8ea23ce7-0077-4e03-8a53-76b38b05c13d"). InnerVolumeSpecName "kube-api-access-zv6q4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:44:10.104779 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.104749    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb8946d9-1328-4dd2-9190-539162c4b377-kube-api-access-d22gp" (OuterVolumeSpecName: "kube-api-access-d22gp") pod "bb8946d9-1328-4dd2-9190-539162c4b377" (UID: "bb8946d9-1328-4dd2-9190-539162c4b377"). InnerVolumeSpecName "kube-api-access-d22gp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:44:10.152947 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.152872    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:10.204174 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.204138    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv6q4\" (UniqueName: \"kubernetes.io/projected/8ea23ce7-0077-4e03-8a53-76b38b05c13d-kube-api-access-zv6q4\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:44:10.204174 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.204175    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8946d9-1328-4dd2-9190-539162c4b377-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:44:10.204384 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.204191    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea23ce7-0077-4e03-8a53-76b38b05c13d-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:44:10.204384 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.204203    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d22gp\" (UniqueName: \"kubernetes.io/projected/bb8946d9-1328-4dd2-9190-539162c4b377-kube-api-access-d22gp\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:44:10.641020 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.640983    2556 generic.go:356] "Generic (PLEG): container finished" podID="bb8946d9-1328-4dd2-9190-539162c4b377" containerID="61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6" exitCode=0
Jul 29 18:44:10.641620 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.641033    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"
Jul 29 18:44:10.641620 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.641067    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49" event={"ID":"bb8946d9-1328-4dd2-9190-539162c4b377","Type":"ContainerDied","Data":"61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6"}
Jul 29 18:44:10.641620 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.641125    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49" event={"ID":"bb8946d9-1328-4dd2-9190-539162c4b377","Type":"ContainerDied","Data":"288bbeef72397596251691667fa8e252895b1772d070a0b7f65ab8442d48b0d5"}
Jul 29 18:44:10.641620 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.641156    2556 scope.go:117] "RemoveContainer" containerID="61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6"
Jul 29 18:44:10.642357 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.642243    2556 generic.go:356] "Generic (PLEG): container finished" podID="8ea23ce7-0077-4e03-8a53-76b38b05c13d" containerID="6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707" exitCode=0
Jul 29 18:44:10.642357 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.642290    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"
Jul 29 18:44:10.642357 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.642294    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb" event={"ID":"8ea23ce7-0077-4e03-8a53-76b38b05c13d","Type":"ContainerDied","Data":"6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707"}
Jul 29 18:44:10.642512 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.642408    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb" event={"ID":"8ea23ce7-0077-4e03-8a53-76b38b05c13d","Type":"ContainerDied","Data":"8493babc7d0551fc90416886600c3361ce13b5f6e0d771e5c84aff688e5e7394"}
Jul 29 18:44:10.649114 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.649094    2556 scope.go:117] "RemoveContainer" containerID="61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6"
Jul 29 18:44:10.649352 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:44:10.649335    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6\": container with ID starting with 61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6 not found: ID does not exist" containerID="61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6"
Jul 29 18:44:10.649444 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.649372    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6"} err="failed to get container status \"61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6\": rpc error: code = NotFound desc = could not find container \"61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6\": container with ID starting with 61ee65431d7feff74f4eff75906c4561e3f900116ed852943300c77735fb29b6 not found: ID does not exist"
Jul 29 18:44:10.649444 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.649392    2556 scope.go:117] "RemoveContainer" containerID="6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707"
Jul 29 18:44:10.657580 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.657563    2556 scope.go:117] "RemoveContainer" containerID="6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707"
Jul 29 18:44:10.657811 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:44:10.657791    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707\": container with ID starting with 6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707 not found: ID does not exist" containerID="6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707"
Jul 29 18:44:10.657871 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.657815    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707"} err="failed to get container status \"6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707\": rpc error: code = NotFound desc = could not find container \"6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707\": container with ID starting with 6e89617a1bff5baf1d569f246b52bb63d8070629b476ef0d0551cdc24fb10707 not found: ID does not exist"
Jul 29 18:44:10.666111 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.666092    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"]
Jul 29 18:44:10.669372 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.669344    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-109296-5c4b486f46-5tv49"]
Jul 29 18:44:10.680438 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.680356    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"]
Jul 29 18:44:10.682531 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:10.682505    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-109296-6b4fff95dd-rbddb"]
Jul 29 18:44:11.032480 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:11.032398    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:11.078787 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:11.078758    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ea23ce7-0077-4e03-8a53-76b38b05c13d" path="/var/lib/kubelet/pods/8ea23ce7-0077-4e03-8a53-76b38b05c13d/volumes"
Jul 29 18:44:11.079093 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:11.079081    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb8946d9-1328-4dd2-9190-539162c4b377" path="/var/lib/kubelet/pods/bb8946d9-1328-4dd2-9190-539162c4b377/volumes"
Jul 29 18:44:15.791636 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:15.791604    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:15.848467 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:15.848436    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:16.043398 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:16.043295    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:16.269118 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:16.269082    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:31.797069 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:31.797035    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:31.822868 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:31.822839    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"]
Jul 29 18:44:31.823088 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:31.823054    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8" podUID="02edcbe6-2b55-4296-9fb3-7d34a62d68b1" containerName="payload-processing" containerID="cri-o://fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157" gracePeriod=30
Jul 29 18:44:31.829584 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:31.829555    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"]
Jul 29 18:44:31.830067 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:31.829766    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8" podUID="a8c1ea99-6c41-47fe-9705-93832131ed1b" containerName="payload-pre-processing" containerID="cri-o://c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02" gracePeriod=30
Jul 29 18:44:31.864038 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:31.864006    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:31.924067 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:31.924033    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:31.987569 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:31.987496    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:32.095124 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.095100    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:44:32.097821 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.097800    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:44:32.181101 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.181069    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxnz8\" (UniqueName: \"kubernetes.io/projected/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-kube-api-access-dxnz8\") pod \"02edcbe6-2b55-4296-9fb3-7d34a62d68b1\" (UID: \"02edcbe6-2b55-4296-9fb3-7d34a62d68b1\") "
Jul 29 18:44:32.181300 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.181161    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8c1ea99-6c41-47fe-9705-93832131ed1b-plugins-config-volume\") pod \"a8c1ea99-6c41-47fe-9705-93832131ed1b\" (UID: \"a8c1ea99-6c41-47fe-9705-93832131ed1b\") "
Jul 29 18:44:32.181300 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.181183    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-plugins-config-volume\") pod \"02edcbe6-2b55-4296-9fb3-7d34a62d68b1\" (UID: \"02edcbe6-2b55-4296-9fb3-7d34a62d68b1\") "
Jul 29 18:44:32.181300 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.181215    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvg58\" (UniqueName: \"kubernetes.io/projected/a8c1ea99-6c41-47fe-9705-93832131ed1b-kube-api-access-mvg58\") pod \"a8c1ea99-6c41-47fe-9705-93832131ed1b\" (UID: \"a8c1ea99-6c41-47fe-9705-93832131ed1b\") "
Jul 29 18:44:32.181545 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.181519    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8c1ea99-6c41-47fe-9705-93832131ed1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8c1ea99-6c41-47fe-9705-93832131ed1b" (UID: "a8c1ea99-6c41-47fe-9705-93832131ed1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:44:32.181600 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.181543    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "02edcbe6-2b55-4296-9fb3-7d34a62d68b1" (UID: "02edcbe6-2b55-4296-9fb3-7d34a62d68b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:44:32.183257 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.183235    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-kube-api-access-dxnz8" (OuterVolumeSpecName: "kube-api-access-dxnz8") pod "02edcbe6-2b55-4296-9fb3-7d34a62d68b1" (UID: "02edcbe6-2b55-4296-9fb3-7d34a62d68b1"). InnerVolumeSpecName "kube-api-access-dxnz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:44:32.183327 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.183269    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c1ea99-6c41-47fe-9705-93832131ed1b-kube-api-access-mvg58" (OuterVolumeSpecName: "kube-api-access-mvg58") pod "a8c1ea99-6c41-47fe-9705-93832131ed1b" (UID: "a8c1ea99-6c41-47fe-9705-93832131ed1b"). InnerVolumeSpecName "kube-api-access-mvg58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:44:32.282664 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.282634    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8c1ea99-6c41-47fe-9705-93832131ed1b-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:44:32.282664 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.282658    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:44:32.282664 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.282668    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvg58\" (UniqueName: \"kubernetes.io/projected/a8c1ea99-6c41-47fe-9705-93832131ed1b-kube-api-access-mvg58\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:44:32.282876 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.282676    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxnz8\" (UniqueName: \"kubernetes.io/projected/02edcbe6-2b55-4296-9fb3-7d34a62d68b1-kube-api-access-dxnz8\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:44:32.339544 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.339472    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:32.738451 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.738420    2556 generic.go:356] "Generic (PLEG): container finished" podID="a8c1ea99-6c41-47fe-9705-93832131ed1b" containerID="c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02" exitCode=0
Jul 29 18:44:32.738605 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.738462    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"
Jul 29 18:44:32.738605 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.738494    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8" event={"ID":"a8c1ea99-6c41-47fe-9705-93832131ed1b","Type":"ContainerDied","Data":"c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02"}
Jul 29 18:44:32.738605 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.738536    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8" event={"ID":"a8c1ea99-6c41-47fe-9705-93832131ed1b","Type":"ContainerDied","Data":"d3ed054d48b7e06311bd26de105598eaf90b56419ab7d711f4ec31b69136e313"}
Jul 29 18:44:32.738605 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.738559    2556 scope.go:117] "RemoveContainer" containerID="c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02"
Jul 29 18:44:32.739771 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.739748    2556 generic.go:356] "Generic (PLEG): container finished" podID="02edcbe6-2b55-4296-9fb3-7d34a62d68b1" containerID="fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157" exitCode=0
Jul 29 18:44:32.739877 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.739786    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"
Jul 29 18:44:32.739877 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.739794    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8" event={"ID":"02edcbe6-2b55-4296-9fb3-7d34a62d68b1","Type":"ContainerDied","Data":"fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157"}
Jul 29 18:44:32.739877 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.739827    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8" event={"ID":"02edcbe6-2b55-4296-9fb3-7d34a62d68b1","Type":"ContainerDied","Data":"9be739b8262387d16b1ae0797e1ed9bf3c33ddca41cf5421c997ef1c0ea4c066"}
Jul 29 18:44:32.747764 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.747749    2556 scope.go:117] "RemoveContainer" containerID="c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02"
Jul 29 18:44:32.748045 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:44:32.748023    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02\": container with ID starting with c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02 not found: ID does not exist" containerID="c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02"
Jul 29 18:44:32.748126 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.748058    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02"} err="failed to get container status \"c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02\": rpc error: code = NotFound desc = could not find container \"c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02\": container with ID starting with c1b671fb6dd86221c3fbdbd821492d4059fbc613e4451bd6f8c9da8f8a825a02 not found: ID does not exist"
Jul 29 18:44:32.748126 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.748080    2556 scope.go:117] "RemoveContainer" containerID="fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157"
Jul 29 18:44:32.755520 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.755502    2556 scope.go:117] "RemoveContainer" containerID="fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157"
Jul 29 18:44:32.755780 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:44:32.755758    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157\": container with ID starting with fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157 not found: ID does not exist" containerID="fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157"
Jul 29 18:44:32.755849 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.755790    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157"} err="failed to get container status \"fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157\": rpc error: code = NotFound desc = could not find container \"fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157\": container with ID starting with fc566f512441bc328ed6b77808cb2741dc79ba968b2b938db4a0bfd056c7a157 not found: ID does not exist"
Jul 29 18:44:32.762389 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.762353    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"]
Jul 29 18:44:32.764632 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.764611    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-95c5f7-787f969956-pwjl8"]
Jul 29 18:44:32.774523 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.774502    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"]
Jul 29 18:44:32.777041 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:32.777021    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-95c5f7-85548c9884-699q8"]
Jul 29 18:44:33.078350 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:33.078271    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02edcbe6-2b55-4296-9fb3-7d34a62d68b1" path="/var/lib/kubelet/pods/02edcbe6-2b55-4296-9fb3-7d34a62d68b1/volumes"
Jul 29 18:44:33.078707 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:33.078641    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8c1ea99-6c41-47fe-9705-93832131ed1b" path="/var/lib/kubelet/pods/a8c1ea99-6c41-47fe-9705-93832131ed1b/volumes"
Jul 29 18:44:38.205399 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:38.205302    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:38.272153 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:38.272120    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:38.473160 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:38.473077    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:38.696571 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:38.696534    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:39.037806 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:39.037773    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:39.347185 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:39.347103    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:39.649861 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:39.649828    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:42.756124 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:42.756089    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:42.855679 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:42.855649    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:43.015510 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.015424    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:43.531222 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.531186    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:43.628308 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.628268    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:43.679454 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679410    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"]
Jul 29 18:44:43.679698 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679685    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb8946d9-1328-4dd2-9190-539162c4b377" containerName="payload-processing"
Jul 29 18:44:43.679698 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679698    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb8946d9-1328-4dd2-9190-539162c4b377" containerName="payload-processing"
Jul 29 18:44:43.679832 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679717    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ea23ce7-0077-4e03-8a53-76b38b05c13d" containerName="payload-pre-processing"
Jul 29 18:44:43.679832 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679722    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ea23ce7-0077-4e03-8a53-76b38b05c13d" containerName="payload-pre-processing"
Jul 29 18:44:43.679832 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679729    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02edcbe6-2b55-4296-9fb3-7d34a62d68b1" containerName="payload-processing"
Jul 29 18:44:43.679832 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679734    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="02edcbe6-2b55-4296-9fb3-7d34a62d68b1" containerName="payload-processing"
Jul 29 18:44:43.679832 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679740    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8c1ea99-6c41-47fe-9705-93832131ed1b" containerName="payload-pre-processing"
Jul 29 18:44:43.679832 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679744    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8c1ea99-6c41-47fe-9705-93832131ed1b" containerName="payload-pre-processing"
Jul 29 18:44:43.679832 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679795    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb8946d9-1328-4dd2-9190-539162c4b377" containerName="payload-processing"
Jul 29 18:44:43.679832 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679803    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="02edcbe6-2b55-4296-9fb3-7d34a62d68b1" containerName="payload-processing"
Jul 29 18:44:43.679832 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679810    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ea23ce7-0077-4e03-8a53-76b38b05c13d" containerName="payload-pre-processing"
Jul 29 18:44:43.679832 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.679816    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8c1ea99-6c41-47fe-9705-93832131ed1b" containerName="payload-pre-processing"
Jul 29 18:44:43.684054 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.684037    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 18:44:43.686697 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.686670    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-6e9884\""
Jul 29 18:44:43.686813 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.686705    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-6e9884-dockercfg-nd77n\""
Jul 29 18:44:43.691751 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.691727    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"]
Jul 29 18:44:43.710072 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.710045    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:44:43.716551 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.716520    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"]
Jul 29 18:44:43.719609 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.719592    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 18:44:43.731237 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.731211    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"]
Jul 29 18:44:43.781438 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.781345    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjdgk\" (UniqueName: \"kubernetes.io/projected/b8a934b6-fb6a-49fa-b336-f05c60c74642-kube-api-access-hjdgk\") pod \"payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht\" (UID: \"b8a934b6-fb6a-49fa-b336-f05c60c74642\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 18:44:43.781793 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.781439    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8a934b6-fb6a-49fa-b336-f05c60c74642-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht\" (UID: \"b8a934b6-fb6a-49fa-b336-f05c60c74642\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 18:44:43.882160 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.882123    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjdgk\" (UniqueName: \"kubernetes.io/projected/b8a934b6-fb6a-49fa-b336-f05c60c74642-kube-api-access-hjdgk\") pod \"payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht\" (UID: \"b8a934b6-fb6a-49fa-b336-f05c60c74642\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 18:44:43.882160 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.882163    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8a934b6-fb6a-49fa-b336-f05c60c74642-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht\" (UID: \"b8a934b6-fb6a-49fa-b336-f05c60c74642\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 18:44:43.882398 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.882249    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbkhd\" (UniqueName: \"kubernetes.io/projected/0665d60e-fe13-4f4a-9b52-c88699847c96-kube-api-access-wbkhd\") pod \"payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk\" (UID: \"0665d60e-fe13-4f4a-9b52-c88699847c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 18:44:43.882398 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.882297    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0665d60e-fe13-4f4a-9b52-c88699847c96-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk\" (UID: \"0665d60e-fe13-4f4a-9b52-c88699847c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 18:44:43.882932 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.882915    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8a934b6-fb6a-49fa-b336-f05c60c74642-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht\" (UID: \"b8a934b6-fb6a-49fa-b336-f05c60c74642\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 18:44:43.892282 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.892256    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjdgk\" (UniqueName: \"kubernetes.io/projected/b8a934b6-fb6a-49fa-b336-f05c60c74642-kube-api-access-hjdgk\") pod \"payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht\" (UID: \"b8a934b6-fb6a-49fa-b336-f05c60c74642\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 18:44:43.983491 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.983453    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbkhd\" (UniqueName: \"kubernetes.io/projected/0665d60e-fe13-4f4a-9b52-c88699847c96-kube-api-access-wbkhd\") pod \"payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk\" (UID: \"0665d60e-fe13-4f4a-9b52-c88699847c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 18:44:43.983491 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.983491    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0665d60e-fe13-4f4a-9b52-c88699847c96-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk\" (UID: \"0665d60e-fe13-4f4a-9b52-c88699847c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 18:44:43.984107 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.984087    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0665d60e-fe13-4f4a-9b52-c88699847c96-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk\" (UID: \"0665d60e-fe13-4f4a-9b52-c88699847c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 18:44:43.990999 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.990975    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbkhd\" (UniqueName: \"kubernetes.io/projected/0665d60e-fe13-4f4a-9b52-c88699847c96-kube-api-access-wbkhd\") pod \"payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk\" (UID: \"0665d60e-fe13-4f4a-9b52-c88699847c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 18:44:43.994796 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:43.994781    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 18:44:44.029137 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:44.029107    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 18:44:44.147527 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:44.147482    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"]
Jul 29 18:44:44.149717 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:44:44.149681    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a934b6_fb6a_49fa_b336_f05c60c74642.slice/crio-c97c29fbf57c284e13fb281c841e5e60f7ef7ad055df9fdc43cbe38a6c50d1e1 WatchSource:0}: Error finding container c97c29fbf57c284e13fb281c841e5e60f7ef7ad055df9fdc43cbe38a6c50d1e1: Status 404 returned error can't find the container with id c97c29fbf57c284e13fb281c841e5e60f7ef7ad055df9fdc43cbe38a6c50d1e1
Jul 29 18:44:44.184974 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:44.184943    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"]
Jul 29 18:44:44.188012 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:44:44.187989    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0665d60e_fe13_4f4a_9b52_c88699847c96.slice/crio-e804406c1649790b0acc25d39f26dd1f53161dfee666ed30c541403e27ff2c19 WatchSource:0}: Error finding container e804406c1649790b0acc25d39f26dd1f53161dfee666ed30c541403e27ff2c19: Status 404 returned error can't find the container with id e804406c1649790b0acc25d39f26dd1f53161dfee666ed30c541403e27ff2c19
Jul 29 18:44:44.795011 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:44.794973    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk" event={"ID":"0665d60e-fe13-4f4a-9b52-c88699847c96","Type":"ContainerStarted","Data":"671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668"}
Jul 29 18:44:44.795457 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:44.795032    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk" event={"ID":"0665d60e-fe13-4f4a-9b52-c88699847c96","Type":"ContainerStarted","Data":"e804406c1649790b0acc25d39f26dd1f53161dfee666ed30c541403e27ff2c19"}
Jul 29 18:44:44.795457 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:44.795087    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 18:44:44.796508 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:44.796490    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht" event={"ID":"b8a934b6-fb6a-49fa-b336-f05c60c74642","Type":"ContainerStarted","Data":"6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d"}
Jul 29 18:44:44.796595 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:44.796534    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht" event={"ID":"b8a934b6-fb6a-49fa-b336-f05c60c74642","Type":"ContainerStarted","Data":"c97c29fbf57c284e13fb281c841e5e60f7ef7ad055df9fdc43cbe38a6c50d1e1"}
Jul 29 18:44:44.796595 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:44.796555    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 18:44:44.810146 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:44.810103    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk" podStartSLOduration=1.8100900279999999 podStartE2EDuration="1.810090028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:44:44.809695639 +0000 UTC m=+3154.318079230" watchObservedRunningTime="2026-07-29 18:44:44.810090028 +0000 UTC m=+3154.318473616"
Jul 29 18:44:44.825454 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:44.825405    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht" podStartSLOduration=1.82538888 podStartE2EDuration="1.82538888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:44:44.823923809 +0000 UTC m=+3154.332307400" watchObservedRunningTime="2026-07-29 18:44:44.82538888 +0000 UTC m=+3154.333772469"
Jul 29 18:44:55.803524 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:55.803489    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 18:44:55.803903 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:44:55.803658    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 18:45:08.222291 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:08.222255    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:45:09.553119 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:09.553092    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:45:10.914830 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:10.914803    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:45:11.012420 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.012387    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:45:11.128335 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.128305    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:45:11.461820 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.461783    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:45:11.577985 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.577910    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:45:11.678282 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.678247    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:45:11.687280 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.687253    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"]
Jul 29 18:45:11.691864 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.691846    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 18:45:11.698643 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.698624    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-750f0d-dockercfg-p7mdv\""
Jul 29 18:45:11.699503 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.699473    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-750f0d\""
Jul 29 18:45:11.704078 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.704043    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"]
Jul 29 18:45:11.719464 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.719438    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"]
Jul 29 18:45:11.722610 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.722592    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 18:45:11.724890 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.724866    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b8mr\" (UniqueName: \"kubernetes.io/projected/515d7540-f685-42f7-815a-85d53aa3969d-kube-api-access-4b8mr\") pod \"payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5\" (UID: \"515d7540-f685-42f7-815a-85d53aa3969d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 18:45:11.724993 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.724917    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/515d7540-f685-42f7-815a-85d53aa3969d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5\" (UID: \"515d7540-f685-42f7-815a-85d53aa3969d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 18:45:11.732275 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.732248    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"]
Jul 29 18:45:11.825796 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.825764    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/515d7540-f685-42f7-815a-85d53aa3969d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5\" (UID: \"515d7540-f685-42f7-815a-85d53aa3969d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 18:45:11.825961 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.825810    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j\" (UID: \"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 18:45:11.825961 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.825852    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b8mr\" (UniqueName: \"kubernetes.io/projected/515d7540-f685-42f7-815a-85d53aa3969d-kube-api-access-4b8mr\") pod \"payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5\" (UID: \"515d7540-f685-42f7-815a-85d53aa3969d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 18:45:11.825961 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.825874    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr8vw\" (UniqueName: \"kubernetes.io/projected/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-kube-api-access-cr8vw\") pod \"payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j\" (UID: \"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 18:45:11.826633 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.826608    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/515d7540-f685-42f7-815a-85d53aa3969d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5\" (UID: \"515d7540-f685-42f7-815a-85d53aa3969d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 18:45:11.834038 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.833980    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b8mr\" (UniqueName: \"kubernetes.io/projected/515d7540-f685-42f7-815a-85d53aa3969d-kube-api-access-4b8mr\") pod \"payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5\" (UID: \"515d7540-f685-42f7-815a-85d53aa3969d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 18:45:11.926750 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.926722    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr8vw\" (UniqueName: \"kubernetes.io/projected/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-kube-api-access-cr8vw\") pod \"payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j\" (UID: \"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 18:45:11.927103 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.926794    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j\" (UID: \"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 18:45:11.927496 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.927475    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j\" (UID: \"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 18:45:11.935118 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:11.935092    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr8vw\" (UniqueName: \"kubernetes.io/projected/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-kube-api-access-cr8vw\") pod \"payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j\" (UID: \"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 18:45:12.001331 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.001301    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 18:45:12.032676 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.032646    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 18:45:12.144272 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.144235    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"]
Jul 29 18:45:12.150204 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:45:12.150170    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod515d7540_f685_42f7_815a_85d53aa3969d.slice/crio-605b8f20fb205f82380a24939347b63561e3f8ffd243730477a83b347864b2b0 WatchSource:0}: Error finding container 605b8f20fb205f82380a24939347b63561e3f8ffd243730477a83b347864b2b0: Status 404 returned error can't find the container with id 605b8f20fb205f82380a24939347b63561e3f8ffd243730477a83b347864b2b0
Jul 29 18:45:12.172093 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.172073    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"]
Jul 29 18:45:12.189491 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:45:12.189464    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae94ec0_f0bb_4d3b_a75d_0b91491881b9.slice/crio-3e51b8dcd71d7797d2c6c5234a4dc68cc11c31813b93b383a41adf01859d17b6 WatchSource:0}: Error finding container 3e51b8dcd71d7797d2c6c5234a4dc68cc11c31813b93b383a41adf01859d17b6: Status 404 returned error can't find the container with id 3e51b8dcd71d7797d2c6c5234a4dc68cc11c31813b93b383a41adf01859d17b6
Jul 29 18:45:12.915719 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.915680    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j" event={"ID":"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9","Type":"ContainerStarted","Data":"d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964"}
Jul 29 18:45:12.915907 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.915738    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j" event={"ID":"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9","Type":"ContainerStarted","Data":"3e51b8dcd71d7797d2c6c5234a4dc68cc11c31813b93b383a41adf01859d17b6"}
Jul 29 18:45:12.915907 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.915800    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 18:45:12.917171 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.917140    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5" event={"ID":"515d7540-f685-42f7-815a-85d53aa3969d","Type":"ContainerStarted","Data":"4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5"}
Jul 29 18:45:12.917296 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.917197    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 18:45:12.917296 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.917216    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5" event={"ID":"515d7540-f685-42f7-815a-85d53aa3969d","Type":"ContainerStarted","Data":"605b8f20fb205f82380a24939347b63561e3f8ffd243730477a83b347864b2b0"}
Jul 29 18:45:12.933242 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.933193    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j" podStartSLOduration=1.9331741230000001 podStartE2EDuration="1.933174123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:45:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:45:12.930131773 +0000 UTC m=+3182.438515360" watchObservedRunningTime="2026-07-29 18:45:12.933174123 +0000 UTC m=+3182.441557716"
Jul 29 18:45:12.951321 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:12.951275    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5" podStartSLOduration=1.9512614419999998 podStartE2EDuration="1.951261442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:45:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:45:12.949522304 +0000 UTC m=+3182.457905891" watchObservedRunningTime="2026-07-29 18:45:12.951261442 +0000 UTC m=+3182.459645029"
Jul 29 18:45:23.923157 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:23.923127    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 18:45:23.923743 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:23.923214    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 18:45:35.878614 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:35.878579    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:45:56.123231 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.123200    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"]
Jul 29 18:45:56.126699 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.126676    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.129096 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.129074    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6e9884\"/\"openshift-service-ca.crt\""
Jul 29 18:45:56.130074 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.130054    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6e9884\"/\"default-dockercfg-2lqcq\""
Jul 29 18:45:56.130182 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.130055    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6e9884\"/\"kube-root-ca.crt\""
Jul 29 18:45:56.130182 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.130084    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6e9884\"/\"auth-test-model-6e9884-kserve-self-signed-certs\""
Jul 29 18:45:56.135767 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.135744    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"]
Jul 29 18:45:56.155522 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.155494    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:45:56.298549 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.298505    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-home\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.298549 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.298547    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k587\" (UniqueName: \"kubernetes.io/projected/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-kube-api-access-4k587\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.298808 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.298575    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-model-cache\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.298808 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.298666    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tmp-dir\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.298808 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.298767    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tls-certs\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.298808 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.298805    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-dshm\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.399452 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.399421    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tls-certs\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.399622 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.399463    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-dshm\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.399622 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.399490    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-home\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.399622 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.399506    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k587\" (UniqueName: \"kubernetes.io/projected/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-kube-api-access-4k587\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.399622 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.399564    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-model-cache\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.399622 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.399598    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tmp-dir\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.400085 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.400059    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-home\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.400214 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.400130    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-model-cache\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.400214 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.400086    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tmp-dir\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.402040 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.402018    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-dshm\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.402242 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.402220    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tls-certs\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.408166 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.408143    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k587\" (UniqueName: \"kubernetes.io/projected/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-kube-api-access-4k587\") pod \"auth-test-model-6e9884-kserve-569657bb87-wrl55\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") " pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.437864 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.437841    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:45:56.565923 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:56.565896    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"]
Jul 29 18:45:56.567810 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:45:56.567781    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cbfcfde_cb98_4a2a_959d_f7f5a1461eb5.slice/crio-1d4c9a2ce0d5124a015f75b847a6cdde8d46289a900bebfe0a35e6056db3d891 WatchSource:0}: Error finding container 1d4c9a2ce0d5124a015f75b847a6cdde8d46289a900bebfe0a35e6056db3d891: Status 404 returned error can't find the container with id 1d4c9a2ce0d5124a015f75b847a6cdde8d46289a900bebfe0a35e6056db3d891
Jul 29 18:45:57.100996 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:57.100959    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55" event={"ID":"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5","Type":"ContainerStarted","Data":"3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68"}
Jul 29 18:45:57.100996 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:57.100999    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55" event={"ID":"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5","Type":"ContainerStarted","Data":"1d4c9a2ce0d5124a015f75b847a6cdde8d46289a900bebfe0a35e6056db3d891"}
Jul 29 18:45:57.116627 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:45:57.116578    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55" podStartSLOduration=1.116562648 podStartE2EDuration="1.116562648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:45:57.11631328 +0000 UTC m=+3226.624696868" watchObservedRunningTime="2026-07-29 18:45:57.116562648 +0000 UTC m=+3226.624946235"
Jul 29 18:46:06.438525 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:46:06.438488    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:46:06.438997 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:46:06.438638    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:46:06.451284 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:46:06.451257    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:46:07.152940 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:46:07.152910    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:46:21.760472 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:46:21.760435    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:46:49.228392 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:46:49.228350    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:48:42.009605 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:48:42.009558    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:48:42.097123 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:48:42.097095    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:50:14.042906 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:50:14.042871    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:50:58.274293 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:50:58.274216    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:51:12.657849 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:12.657811    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"]
Jul 29 18:51:12.681403 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:12.681378    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:51:13.526921 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:13.526891    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:51:24.918017 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:24.917972    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:51:27.676458 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.676420    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55" podUID="4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" containerName="main" containerID="cri-o://3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68" gracePeriod=45
Jul 29 18:51:27.923394 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.923354    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:51:27.990038 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.989959    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-dshm\") pod \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") "
Jul 29 18:51:27.990038 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.989996    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-model-cache\") pod \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") "
Jul 29 18:51:27.990038 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.990032    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4k587\" (UniqueName: \"kubernetes.io/projected/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-kube-api-access-4k587\") pod \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") "
Jul 29 18:51:27.990312 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.990087    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tls-certs\") pod \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") "
Jul 29 18:51:27.990312 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.990124    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tmp-dir\") pod \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") "
Jul 29 18:51:27.990312 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.990163    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-home\") pod \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\" (UID: \"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5\") "
Jul 29 18:51:27.990312 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.990284    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-model-cache" (OuterVolumeSpecName: "model-cache") pod "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" (UID: "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:51:27.990544 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.990398    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" (UID: "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:51:27.990544 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.990445    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-model-cache\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:51:27.990544 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.990483    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-home" (OuterVolumeSpecName: "home") pod "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" (UID: "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:51:27.992315 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.992294    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-dshm" (OuterVolumeSpecName: "dshm") pod "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" (UID: "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:51:27.992434 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.992316    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-kube-api-access-4k587" (OuterVolumeSpecName: "kube-api-access-4k587") pod "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" (UID: "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5"). InnerVolumeSpecName "kube-api-access-4k587". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:51:27.992434 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:27.992337    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" (UID: "4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:51:28.090879 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.090851    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4k587\" (UniqueName: \"kubernetes.io/projected/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-kube-api-access-4k587\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:51:28.090879 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.090878    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tls-certs\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:51:28.090879 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.090888    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-tmp-dir\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:51:28.091088 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.090896    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-home\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:51:28.091088 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.090906    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5-dshm\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:51:28.470601 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.470561    2556 generic.go:356] "Generic (PLEG): container finished" podID="4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" containerID="3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68" exitCode=0
Jul 29 18:51:28.470828 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.470611    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"
Jul 29 18:51:28.470828 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.470652    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55" event={"ID":"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5","Type":"ContainerDied","Data":"3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68"}
Jul 29 18:51:28.470828 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.470707    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55" event={"ID":"4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5","Type":"ContainerDied","Data":"1d4c9a2ce0d5124a015f75b847a6cdde8d46289a900bebfe0a35e6056db3d891"}
Jul 29 18:51:28.470828 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.470730    2556 scope.go:117] "RemoveContainer" containerID="3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68"
Jul 29 18:51:28.481915 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.481892    2556 scope.go:117] "RemoveContainer" containerID="3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68"
Jul 29 18:51:28.482186 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:51:28.482163    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68\": container with ID starting with 3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68 not found: ID does not exist" containerID="3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68"
Jul 29 18:51:28.482286 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.482199    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68"} err="failed to get container status \"3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68\": rpc error: code = NotFound desc = could not find container \"3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68\": container with ID starting with 3b3f49491c14456a3ed4a3eb66a3630452408764ba21f56db97bbe60b7f1af68 not found: ID does not exist"
Jul 29 18:51:28.492821 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.492797    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"]
Jul 29 18:51:28.494981 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:28.494958    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/auth-test-model-6e9884-kserve-569657bb87-wrl55"]
Jul 29 18:51:29.079041 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:29.079012    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" path="/var/lib/kubelet/pods/4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5/volumes"
Jul 29 18:51:30.633124 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.633078    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"]
Jul 29 18:51:30.633704 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.633574    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" containerName="main"
Jul 29 18:51:30.633704 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.633594    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" containerName="main"
Jul 29 18:51:30.633704 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.633669    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cbfcfde-cb98-4a2a-959d-f7f5a1461eb5" containerName="main"
Jul 29 18:51:30.637972 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.637950    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.641008 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.640990    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6e9884\"/\"kube-root-ca.crt\""
Jul 29 18:51:30.642340 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.642319    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6e9884\"/\"default-dockercfg-2lqcq\""
Jul 29 18:51:30.642436 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.642420    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6e9884\"/\"openshift-service-ca.crt\""
Jul 29 18:51:30.642499 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.642421    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6e9884\"/\"sub-test-model-6e9884-kserve-self-signed-certs\""
Jul 29 18:51:30.650067 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.650039    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"]
Jul 29 18:51:30.662549 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.662529    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:51:30.710437 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.710407    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-dshm\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.710567 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.710447    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-home\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.710567 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.710463    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-tmp-dir\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.710567 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.710483    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc6s5\" (UniqueName: \"kubernetes.io/projected/ea4fabc4-3492-4463-9aa5-acaa386e5223-kube-api-access-hc6s5\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.710567 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.710525    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-model-cache\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.710567 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.710542    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4fabc4-3492-4463-9aa5-acaa386e5223-tls-certs\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.811710 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.811674    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-model-cache\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.811710 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.811711    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4fabc4-3492-4463-9aa5-acaa386e5223-tls-certs\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.811922 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.811743    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-dshm\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.811922 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.811766    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-home\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.811922 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.811781    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-tmp-dir\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.811922 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.811801    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc6s5\" (UniqueName: \"kubernetes.io/projected/ea4fabc4-3492-4463-9aa5-acaa386e5223-kube-api-access-hc6s5\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.812138 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.812111    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-model-cache\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.812198 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.812163    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-home\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.812260 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.812236    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-tmp-dir\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.814140 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.814098    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-dshm\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.814313 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.814297    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4fabc4-3492-4463-9aa5-acaa386e5223-tls-certs\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.830772 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.830748    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc6s5\" (UniqueName: \"kubernetes.io/projected/ea4fabc4-3492-4463-9aa5-acaa386e5223-kube-api-access-hc6s5\") pod \"sub-test-model-6e9884-kserve-5fcf49d755-8gfw4\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") " pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:30.949645 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:30.949555    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:31.079070 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:31.079049    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"]
Jul 29 18:51:31.080570 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:51:31.080544    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea4fabc4_3492_4463_9aa5_acaa386e5223.slice/crio-43152f0066e33838ca3d2597b8a2434ee380f15be797c040225ecd87f67135e0 WatchSource:0}: Error finding container 43152f0066e33838ca3d2597b8a2434ee380f15be797c040225ecd87f67135e0: Status 404 returned error can't find the container with id 43152f0066e33838ca3d2597b8a2434ee380f15be797c040225ecd87f67135e0
Jul 29 18:51:31.485468 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:31.485430    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4" event={"ID":"ea4fabc4-3492-4463-9aa5-acaa386e5223","Type":"ContainerStarted","Data":"772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970"}
Jul 29 18:51:31.485468 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:31.485474    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4" event={"ID":"ea4fabc4-3492-4463-9aa5-acaa386e5223","Type":"ContainerStarted","Data":"43152f0066e33838ca3d2597b8a2434ee380f15be797c040225ecd87f67135e0"}
Jul 29 18:51:31.504570 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:31.504521    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4" podStartSLOduration=1.504506139 podStartE2EDuration="1.504506139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:51:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:51:31.503134835 +0000 UTC m=+3561.011518420" watchObservedRunningTime="2026-07-29 18:51:31.504506139 +0000 UTC m=+3561.012889727"
Jul 29 18:51:40.950585 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:40.950537    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:40.951028 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:40.950605    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:40.963074 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:40.963046    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:41.538551 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:41.538521    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:51:56.942754 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:51:56.942721    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:52:47.580999 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:52:47.580955    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:52:47.632009 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:52:47.631978    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:53:34.104744 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:34.104671    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"]
Jul 29 18:53:34.126855 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:34.126826    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:53:34.985041 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:34.985007    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:53:48.510563 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.510530    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"]
Jul 29 18:53:48.513755 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.513734    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.516598 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.516576    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6e9884\"/\"rate-test-model-6e9884-kserve-self-signed-certs\""
Jul 29 18:53:48.522056 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.522032    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"]
Jul 29 18:53:48.527958 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.527931    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:53:48.565262 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.565240    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wj8q\" (UniqueName: \"kubernetes.io/projected/4f41caa5-2193-4ced-8b17-f4abb22ef158-kube-api-access-5wj8q\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.565377 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.565281    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-model-cache\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.565377 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.565300    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-dshm\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.565377 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.565315    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f41caa5-2193-4ced-8b17-f4abb22ef158-tls-certs\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.565493 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.565386    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-home\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.565493 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.565404    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-tmp-dir\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.666165 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.666125    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-model-cache\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.666165 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.666164    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-dshm\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.666465 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.666179    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f41caa5-2193-4ced-8b17-f4abb22ef158-tls-certs\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.666465 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.666212    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-home\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.666465 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.666233    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-tmp-dir\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.666465 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.666286    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wj8q\" (UniqueName: \"kubernetes.io/projected/4f41caa5-2193-4ced-8b17-f4abb22ef158-kube-api-access-5wj8q\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.666722 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.666616    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-model-cache\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.666722 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.666654    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-home\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.666796 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.666749    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-tmp-dir\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.668624 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.668602    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-dshm\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.668828 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.668808    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f41caa5-2193-4ced-8b17-f4abb22ef158-tls-certs\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.674346 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.674326    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wj8q\" (UniqueName: \"kubernetes.io/projected/4f41caa5-2193-4ced-8b17-f4abb22ef158-kube-api-access-5wj8q\") pod \"rate-test-model-6e9884-kserve-6485cd9ff-bdfwp\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") " pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.823460 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.823351    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:48.955471 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:48.955441    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"]
Jul 29 18:53:48.957691 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:53:48.957660    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f41caa5_2193_4ced_8b17_f4abb22ef158.slice/crio-e9a245ac42d0c65646f7237b2feae1922185b1f58bf4f71ddc5d735cb9fc42a4 WatchSource:0}: Error finding container e9a245ac42d0c65646f7237b2feae1922185b1f58bf4f71ddc5d735cb9fc42a4: Status 404 returned error can't find the container with id e9a245ac42d0c65646f7237b2feae1922185b1f58bf4f71ddc5d735cb9fc42a4
Jul 29 18:53:49.060662 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.060627    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp" event={"ID":"4f41caa5-2193-4ced-8b17-f4abb22ef158","Type":"ContainerStarted","Data":"15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95"}
Jul 29 18:53:49.061062 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.060671    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp" event={"ID":"4f41caa5-2193-4ced-8b17-f4abb22ef158","Type":"ContainerStarted","Data":"e9a245ac42d0c65646f7237b2feae1922185b1f58bf4f71ddc5d735cb9fc42a4"}
Jul 29 18:53:49.078509 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.078415    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp" podStartSLOduration=1.078395147 podStartE2EDuration="1.078395147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:53:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:53:49.076343157 +0000 UTC m=+3698.584726746" watchObservedRunningTime="2026-07-29 18:53:49.078395147 +0000 UTC m=+3698.586778739"
Jul 29 18:53:49.123530 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.123488    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4" podUID="ea4fabc4-3492-4463-9aa5-acaa386e5223" containerName="main" containerID="cri-o://772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970" gracePeriod=45
Jul 29 18:53:49.357935 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.357906    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:53:49.474322 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.474279    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4fabc4-3492-4463-9aa5-acaa386e5223-tls-certs\") pod \"ea4fabc4-3492-4463-9aa5-acaa386e5223\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") "
Jul 29 18:53:49.474534 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.474336    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-dshm\") pod \"ea4fabc4-3492-4463-9aa5-acaa386e5223\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") "
Jul 29 18:53:49.474534 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.474389    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-home\") pod \"ea4fabc4-3492-4463-9aa5-acaa386e5223\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") "
Jul 29 18:53:49.474534 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.474414    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-tmp-dir\") pod \"ea4fabc4-3492-4463-9aa5-acaa386e5223\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") "
Jul 29 18:53:49.474534 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.474466    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc6s5\" (UniqueName: \"kubernetes.io/projected/ea4fabc4-3492-4463-9aa5-acaa386e5223-kube-api-access-hc6s5\") pod \"ea4fabc4-3492-4463-9aa5-acaa386e5223\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") "
Jul 29 18:53:49.474534 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.474515    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-model-cache\") pod \"ea4fabc4-3492-4463-9aa5-acaa386e5223\" (UID: \"ea4fabc4-3492-4463-9aa5-acaa386e5223\") "
Jul 29 18:53:49.474806 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.474665    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-home" (OuterVolumeSpecName: "home") pod "ea4fabc4-3492-4463-9aa5-acaa386e5223" (UID: "ea4fabc4-3492-4463-9aa5-acaa386e5223"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:53:49.474806 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.474739    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ea4fabc4-3492-4463-9aa5-acaa386e5223" (UID: "ea4fabc4-3492-4463-9aa5-acaa386e5223"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:53:49.474898 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.474802    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-home\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:53:49.474898 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.474861    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-model-cache" (OuterVolumeSpecName: "model-cache") pod "ea4fabc4-3492-4463-9aa5-acaa386e5223" (UID: "ea4fabc4-3492-4463-9aa5-acaa386e5223"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:53:49.476501 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.476463    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-dshm" (OuterVolumeSpecName: "dshm") pod "ea4fabc4-3492-4463-9aa5-acaa386e5223" (UID: "ea4fabc4-3492-4463-9aa5-acaa386e5223"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:53:49.476622 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.476502    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea4fabc4-3492-4463-9aa5-acaa386e5223-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ea4fabc4-3492-4463-9aa5-acaa386e5223" (UID: "ea4fabc4-3492-4463-9aa5-acaa386e5223"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:53:49.476622 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.476544    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea4fabc4-3492-4463-9aa5-acaa386e5223-kube-api-access-hc6s5" (OuterVolumeSpecName: "kube-api-access-hc6s5") pod "ea4fabc4-3492-4463-9aa5-acaa386e5223" (UID: "ea4fabc4-3492-4463-9aa5-acaa386e5223"). InnerVolumeSpecName "kube-api-access-hc6s5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:53:49.575534 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.575497    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc6s5\" (UniqueName: \"kubernetes.io/projected/ea4fabc4-3492-4463-9aa5-acaa386e5223-kube-api-access-hc6s5\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:53:49.575534 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.575533    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-model-cache\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:53:49.575534 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.575545    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4fabc4-3492-4463-9aa5-acaa386e5223-tls-certs\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:53:49.575957 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.575553    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-dshm\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:53:49.575957 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:49.575561    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea4fabc4-3492-4463-9aa5-acaa386e5223-tmp-dir\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:53:50.066721 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:50.066634    2556 generic.go:356] "Generic (PLEG): container finished" podID="ea4fabc4-3492-4463-9aa5-acaa386e5223" containerID="772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970" exitCode=0
Jul 29 18:53:50.066721 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:50.066674    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"
Jul 29 18:53:50.066721 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:50.066712    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4" event={"ID":"ea4fabc4-3492-4463-9aa5-acaa386e5223","Type":"ContainerDied","Data":"772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970"}
Jul 29 18:53:50.066983 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:50.066741    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4" event={"ID":"ea4fabc4-3492-4463-9aa5-acaa386e5223","Type":"ContainerDied","Data":"43152f0066e33838ca3d2597b8a2434ee380f15be797c040225ecd87f67135e0"}
Jul 29 18:53:50.066983 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:50.066758    2556 scope.go:117] "RemoveContainer" containerID="772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970"
Jul 29 18:53:50.075135 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:50.075118    2556 scope.go:117] "RemoveContainer" containerID="772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970"
Jul 29 18:53:50.075403 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:53:50.075353    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970\": container with ID starting with 772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970 not found: ID does not exist" containerID="772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970"
Jul 29 18:53:50.075492 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:50.075408    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970"} err="failed to get container status \"772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970\": rpc error: code = NotFound desc = could not find container \"772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970\": container with ID starting with 772352674c35fa791714f0389311b87f8f0a96af840774996b9c80b018b9a970 not found: ID does not exist"
Jul 29 18:53:50.087233 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:50.087202    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"]
Jul 29 18:53:50.091085 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:50.091064    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/sub-test-model-6e9884-kserve-5fcf49d755-8gfw4"]
Jul 29 18:53:51.078892 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:51.078857    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea4fabc4-3492-4463-9aa5-acaa386e5223" path="/var/lib/kubelet/pods/ea4fabc4-3492-4463-9aa5-acaa386e5223/volumes"
Jul 29 18:53:58.824178 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:58.824146    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:58.824586 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:58.824191    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:58.836478 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:58.836454    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:53:59.115604 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:53:59.115529    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:54:22.321940 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.321907    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"]
Jul 29 18:54:22.322338 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.322241    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea4fabc4-3492-4463-9aa5-acaa386e5223" containerName="main"
Jul 29 18:54:22.322338 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.322252    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4fabc4-3492-4463-9aa5-acaa386e5223" containerName="main"
Jul 29 18:54:22.322338 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.322304    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea4fabc4-3492-4463-9aa5-acaa386e5223" containerName="main"
Jul 29 18:54:22.326396 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.326378    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.329201 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.329178    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"openshift-service-ca.crt\""
Jul 29 18:54:22.330566 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.330545    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"kube-root-ca.crt\""
Jul 29 18:54:22.330914 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.330554    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"default-dockercfg-m2gfx\""
Jul 29 18:54:22.330914 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.330580    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"rate-test-model-750f0d-kserve-self-signed-certs\""
Jul 29 18:54:22.333234 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.333207    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"]
Jul 29 18:54:22.342937 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.342896    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:54:22.353349 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.353321    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-home\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.353459 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.353356    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x57lx\" (UniqueName: \"kubernetes.io/projected/3a637802-e211-4a4f-9ac5-ceb77de01e7c-kube-api-access-x57lx\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.353510 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.353474    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-model-cache\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.353510 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.353498    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-dshm\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.353591 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.353517    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tmp-dir\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.353591 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.353548    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tls-certs\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.459427 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.459385    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-model-cache\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.459595 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.459452    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-dshm\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.459644 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.459618    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tmp-dir\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.459895 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.459873    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tls-certs\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.459946 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.459917    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-home\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.459998 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.459947    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x57lx\" (UniqueName: \"kubernetes.io/projected/3a637802-e211-4a4f-9ac5-ceb77de01e7c-kube-api-access-x57lx\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.460102 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.460077    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-model-cache\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.460191 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.460163    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tmp-dir\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.460428 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.460406    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-home\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.462063 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.462030    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-dshm\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.462293 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.462271    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tls-certs\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.468142 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.468115    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x57lx\" (UniqueName: \"kubernetes.io/projected/3a637802-e211-4a4f-9ac5-ceb77de01e7c-kube-api-access-x57lx\") pod \"rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") " pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.636671 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.636640    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:22.766410 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:22.766383    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"]
Jul 29 18:54:22.768865 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:54:22.768837    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a637802_e211_4a4f_9ac5_ceb77de01e7c.slice/crio-e02be1dbef24abbc1d6ab843ca8c25442f6de491d799925853e94c9bc069d972 WatchSource:0}: Error finding container e02be1dbef24abbc1d6ab843ca8c25442f6de491d799925853e94c9bc069d972: Status 404 returned error can't find the container with id e02be1dbef24abbc1d6ab843ca8c25442f6de491d799925853e94c9bc069d972
Jul 29 18:54:23.204567 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:23.204532    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d" event={"ID":"3a637802-e211-4a4f-9ac5-ceb77de01e7c","Type":"ContainerStarted","Data":"7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad"}
Jul 29 18:54:23.204567 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:23.204573    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d" event={"ID":"3a637802-e211-4a4f-9ac5-ceb77de01e7c","Type":"ContainerStarted","Data":"e02be1dbef24abbc1d6ab843ca8c25442f6de491d799925853e94c9bc069d972"}
Jul 29 18:54:23.224650 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:23.224598    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d" podStartSLOduration=1.22458007 podStartE2EDuration="1.22458007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:54:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:54:23.222869191 +0000 UTC m=+3732.731252792" watchObservedRunningTime="2026-07-29 18:54:23.22458007 +0000 UTC m=+3732.732963657"
Jul 29 18:54:32.636943 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:32.636906    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:32.637479 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:32.636969    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:32.649257 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:32.649232    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:54:33.259460 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:54:33.259432    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:56:02.305405 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:02.305354    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"]
Jul 29 18:56:02.355531 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:02.355493    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:03.209173 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:03.209138    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"]
Jul 29 18:56:03.874878 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:03.874847    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:17.324247 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.324190    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp" podUID="4f41caa5-2193-4ced-8b17-f4abb22ef158" containerName="main" containerID="cri-o://15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95" gracePeriod=45
Jul 29 18:56:17.564295 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.564273    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:56:17.645679 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.645653    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-model-cache\") pod \"4f41caa5-2193-4ced-8b17-f4abb22ef158\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") "
Jul 29 18:56:17.645845 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.645717    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-home\") pod \"4f41caa5-2193-4ced-8b17-f4abb22ef158\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") "
Jul 29 18:56:17.645845 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.645753    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wj8q\" (UniqueName: \"kubernetes.io/projected/4f41caa5-2193-4ced-8b17-f4abb22ef158-kube-api-access-5wj8q\") pod \"4f41caa5-2193-4ced-8b17-f4abb22ef158\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") "
Jul 29 18:56:17.645845 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.645782    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f41caa5-2193-4ced-8b17-f4abb22ef158-tls-certs\") pod \"4f41caa5-2193-4ced-8b17-f4abb22ef158\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") "
Jul 29 18:56:17.645845 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.645812    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-tmp-dir\") pod \"4f41caa5-2193-4ced-8b17-f4abb22ef158\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") "
Jul 29 18:56:17.645845 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.645845    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-dshm\") pod \"4f41caa5-2193-4ced-8b17-f4abb22ef158\" (UID: \"4f41caa5-2193-4ced-8b17-f4abb22ef158\") "
Jul 29 18:56:17.646099 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.645973    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f41caa5-2193-4ced-8b17-f4abb22ef158" (UID: "4f41caa5-2193-4ced-8b17-f4abb22ef158"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:56:17.646099 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.646031    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-home" (OuterVolumeSpecName: "home") pod "4f41caa5-2193-4ced-8b17-f4abb22ef158" (UID: "4f41caa5-2193-4ced-8b17-f4abb22ef158"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:56:17.646179 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.646111    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f41caa5-2193-4ced-8b17-f4abb22ef158" (UID: "4f41caa5-2193-4ced-8b17-f4abb22ef158"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:56:17.647905 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.647884    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-dshm" (OuterVolumeSpecName: "dshm") pod "4f41caa5-2193-4ced-8b17-f4abb22ef158" (UID: "4f41caa5-2193-4ced-8b17-f4abb22ef158"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:56:17.648022 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.647940    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f41caa5-2193-4ced-8b17-f4abb22ef158-kube-api-access-5wj8q" (OuterVolumeSpecName: "kube-api-access-5wj8q") pod "4f41caa5-2193-4ced-8b17-f4abb22ef158" (UID: "4f41caa5-2193-4ced-8b17-f4abb22ef158"). InnerVolumeSpecName "kube-api-access-5wj8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:56:17.648022 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.647947    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f41caa5-2193-4ced-8b17-f4abb22ef158-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f41caa5-2193-4ced-8b17-f4abb22ef158" (UID: "4f41caa5-2193-4ced-8b17-f4abb22ef158"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:56:17.681240 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.681212    2556 generic.go:356] "Generic (PLEG): container finished" podID="4f41caa5-2193-4ced-8b17-f4abb22ef158" containerID="15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95" exitCode=0
Jul 29 18:56:17.681387 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.681251    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"
Jul 29 18:56:17.681387 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.681262    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp" event={"ID":"4f41caa5-2193-4ced-8b17-f4abb22ef158","Type":"ContainerDied","Data":"15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95"}
Jul 29 18:56:17.681387 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.681287    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp" event={"ID":"4f41caa5-2193-4ced-8b17-f4abb22ef158","Type":"ContainerDied","Data":"e9a245ac42d0c65646f7237b2feae1922185b1f58bf4f71ddc5d735cb9fc42a4"}
Jul 29 18:56:17.681387 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.681303    2556 scope.go:117] "RemoveContainer" containerID="15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95"
Jul 29 18:56:17.689710 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.689693    2556 scope.go:117] "RemoveContainer" containerID="15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95"
Jul 29 18:56:17.689931 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:56:17.689912    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95\": container with ID starting with 15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95 not found: ID does not exist" containerID="15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95"
Jul 29 18:56:17.689978 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.689939    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95"} err="failed to get container status \"15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95\": rpc error: code = NotFound desc = could not find container \"15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95\": container with ID starting with 15719dc4b5c2aaee56ce4561cd6e746ed56b8839fac09829a9e36aa77e602e95 not found: ID does not exist"
Jul 29 18:56:17.703200 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.703175    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"]
Jul 29 18:56:17.706936 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.706916    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6e9884/rate-test-model-6e9884-kserve-6485cd9ff-bdfwp"]
Jul 29 18:56:17.746646 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.746622    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-home\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:17.746646 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.746644    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wj8q\" (UniqueName: \"kubernetes.io/projected/4f41caa5-2193-4ced-8b17-f4abb22ef158-kube-api-access-5wj8q\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:17.746781 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.746654    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f41caa5-2193-4ced-8b17-f4abb22ef158-tls-certs\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:17.746781 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.746662    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-tmp-dir\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:17.746781 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.746670    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-dshm\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:17.746781 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:17.746677    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f41caa5-2193-4ced-8b17-f4abb22ef158-model-cache\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:18.227836 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.227792    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d" podUID="3a637802-e211-4a4f-9ac5-ceb77de01e7c" containerName="main" containerID="cri-o://7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad" gracePeriod=45
Jul 29 18:56:18.480612 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.480549    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:56:18.552859 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.552830    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-home\") pod \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") "
Jul 29 18:56:18.552859 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.552859    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-model-cache\") pod \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") "
Jul 29 18:56:18.553095 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.552900    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x57lx\" (UniqueName: \"kubernetes.io/projected/3a637802-e211-4a4f-9ac5-ceb77de01e7c-kube-api-access-x57lx\") pod \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") "
Jul 29 18:56:18.553095 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.552946    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-dshm\") pod \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") "
Jul 29 18:56:18.553095 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.552963    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tmp-dir\") pod \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") "
Jul 29 18:56:18.553095 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.552984    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tls-certs\") pod \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\" (UID: \"3a637802-e211-4a4f-9ac5-ceb77de01e7c\") "
Jul 29 18:56:18.553287 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.553124    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-home" (OuterVolumeSpecName: "home") pod "3a637802-e211-4a4f-9ac5-ceb77de01e7c" (UID: "3a637802-e211-4a4f-9ac5-ceb77de01e7c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:56:18.553287 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.553139    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-model-cache" (OuterVolumeSpecName: "model-cache") pod "3a637802-e211-4a4f-9ac5-ceb77de01e7c" (UID: "3a637802-e211-4a4f-9ac5-ceb77de01e7c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:56:18.553287 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.553206    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3a637802-e211-4a4f-9ac5-ceb77de01e7c" (UID: "3a637802-e211-4a4f-9ac5-ceb77de01e7c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:56:18.553471 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.553333    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tmp-dir\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:18.553471 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.553354    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-home\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:18.553471 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.553390    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-model-cache\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:18.555256 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.555227    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-dshm" (OuterVolumeSpecName: "dshm") pod "3a637802-e211-4a4f-9ac5-ceb77de01e7c" (UID: "3a637802-e211-4a4f-9ac5-ceb77de01e7c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:56:18.555256 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.555230    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3a637802-e211-4a4f-9ac5-ceb77de01e7c" (UID: "3a637802-e211-4a4f-9ac5-ceb77de01e7c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:56:18.555435 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.555287    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a637802-e211-4a4f-9ac5-ceb77de01e7c-kube-api-access-x57lx" (OuterVolumeSpecName: "kube-api-access-x57lx") pod "3a637802-e211-4a4f-9ac5-ceb77de01e7c" (UID: "3a637802-e211-4a4f-9ac5-ceb77de01e7c"). InnerVolumeSpecName "kube-api-access-x57lx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:56:18.654750 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.654711    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x57lx\" (UniqueName: \"kubernetes.io/projected/3a637802-e211-4a4f-9ac5-ceb77de01e7c-kube-api-access-x57lx\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:18.654750 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.654741    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a637802-e211-4a4f-9ac5-ceb77de01e7c-dshm\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:18.654750 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.654756    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a637802-e211-4a4f-9ac5-ceb77de01e7c-tls-certs\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:56:18.687356 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.687323    2556 generic.go:356] "Generic (PLEG): container finished" podID="3a637802-e211-4a4f-9ac5-ceb77de01e7c" containerID="7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad" exitCode=0
Jul 29 18:56:18.687514 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.687382    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"
Jul 29 18:56:18.687514 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.687405    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d" event={"ID":"3a637802-e211-4a4f-9ac5-ceb77de01e7c","Type":"ContainerDied","Data":"7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad"}
Jul 29 18:56:18.687514 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.687439    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d" event={"ID":"3a637802-e211-4a4f-9ac5-ceb77de01e7c","Type":"ContainerDied","Data":"e02be1dbef24abbc1d6ab843ca8c25442f6de491d799925853e94c9bc069d972"}
Jul 29 18:56:18.687514 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.687459    2556 scope.go:117] "RemoveContainer" containerID="7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad"
Jul 29 18:56:18.695983 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.695964    2556 scope.go:117] "RemoveContainer" containerID="7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad"
Jul 29 18:56:18.696215 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:56:18.696195    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad\": container with ID starting with 7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad not found: ID does not exist" containerID="7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad"
Jul 29 18:56:18.696286 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.696222    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad"} err="failed to get container status \"7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad\": rpc error: code = NotFound desc = could not find container \"7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad\": container with ID starting with 7234565c39d398fd76449971412dd44041b2df95fb1426ccf653d8da071d19ad not found: ID does not exist"
Jul 29 18:56:18.707924 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.707901    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"]
Jul 29 18:56:18.711713 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:18.711683    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/rate-test-model-750f0d-kserve-54df9bdbdc-jbl9d"]
Jul 29 18:56:19.078154 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:19.078120    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a637802-e211-4a4f-9ac5-ceb77de01e7c" path="/var/lib/kubelet/pods/3a637802-e211-4a4f-9ac5-ceb77de01e7c/volumes"
Jul 29 18:56:19.078521 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:19.078507    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f41caa5-2193-4ced-8b17-f4abb22ef158" path="/var/lib/kubelet/pods/4f41caa5-2193-4ced-8b17-f4abb22ef158/volumes"
Jul 29 18:56:22.908686 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:22.908646    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:23.213654 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:23.213565    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:23.502154 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:23.502067    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:26.202971 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.202940    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:26.296711 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.296665    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:26.367741 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.367709    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:26.565815 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.565728    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:26.888837 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.888779    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:26.983218 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.983187    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:26.987838 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.987809    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"]
Jul 29 18:56:26.988142 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.988129    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a637802-e211-4a4f-9ac5-ceb77de01e7c" containerName="main"
Jul 29 18:56:26.988188 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.988143    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a637802-e211-4a4f-9ac5-ceb77de01e7c" containerName="main"
Jul 29 18:56:26.988188 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.988156    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f41caa5-2193-4ced-8b17-f4abb22ef158" containerName="main"
Jul 29 18:56:26.988188 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.988162    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f41caa5-2193-4ced-8b17-f4abb22ef158" containerName="main"
Jul 29 18:56:26.988281 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.988219    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f41caa5-2193-4ced-8b17-f4abb22ef158" containerName="main"
Jul 29 18:56:26.988281 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.988226    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a637802-e211-4a4f-9ac5-ceb77de01e7c" containerName="main"
Jul 29 18:56:26.992471 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.992450    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:56:26.994985 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.994966    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-3140cc-dockercfg-dw4vh\""
Jul 29 18:56:26.995274 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:26.995259    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-3140cc\""
Jul 29 18:56:27.002877 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.002854    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"]
Jul 29 18:56:27.029053 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.029028    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"]
Jul 29 18:56:27.032484 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.032463    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:56:27.041074 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.041050    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"]
Jul 29 18:56:27.125854 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.125812    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3af08e09-372f-4fc2-8578-b83397eecfad-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g\" (UID: \"3af08e09-372f-4fc2-8578-b83397eecfad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:56:27.125854 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.125857    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98k6x\" (UniqueName: \"kubernetes.io/projected/8a641cc4-4ae2-4544-aef3-788a4a6606c1-kube-api-access-98k6x\") pod \"payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w\" (UID: \"8a641cc4-4ae2-4544-aef3-788a4a6606c1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:56:27.126121 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.125892    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a641cc4-4ae2-4544-aef3-788a4a6606c1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w\" (UID: \"8a641cc4-4ae2-4544-aef3-788a4a6606c1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:56:27.126121 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.126041    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn5hl\" (UniqueName: \"kubernetes.io/projected/3af08e09-372f-4fc2-8578-b83397eecfad-kube-api-access-rn5hl\") pod \"payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g\" (UID: \"3af08e09-372f-4fc2-8578-b83397eecfad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:56:27.226998 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.226915    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn5hl\" (UniqueName: \"kubernetes.io/projected/3af08e09-372f-4fc2-8578-b83397eecfad-kube-api-access-rn5hl\") pod \"payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g\" (UID: \"3af08e09-372f-4fc2-8578-b83397eecfad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:56:27.226998 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.226956    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3af08e09-372f-4fc2-8578-b83397eecfad-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g\" (UID: \"3af08e09-372f-4fc2-8578-b83397eecfad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:56:27.226998 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.226986    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98k6x\" (UniqueName: \"kubernetes.io/projected/8a641cc4-4ae2-4544-aef3-788a4a6606c1-kube-api-access-98k6x\") pod \"payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w\" (UID: \"8a641cc4-4ae2-4544-aef3-788a4a6606c1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:56:27.227616 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.227019    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a641cc4-4ae2-4544-aef3-788a4a6606c1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w\" (UID: \"8a641cc4-4ae2-4544-aef3-788a4a6606c1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:56:27.227782 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.227760    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a641cc4-4ae2-4544-aef3-788a4a6606c1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w\" (UID: \"8a641cc4-4ae2-4544-aef3-788a4a6606c1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:56:27.227878 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.227854    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3af08e09-372f-4fc2-8578-b83397eecfad-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g\" (UID: \"3af08e09-372f-4fc2-8578-b83397eecfad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:56:27.235474 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.235451    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn5hl\" (UniqueName: \"kubernetes.io/projected/3af08e09-372f-4fc2-8578-b83397eecfad-kube-api-access-rn5hl\") pod \"payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g\" (UID: \"3af08e09-372f-4fc2-8578-b83397eecfad\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:56:27.235728 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.235707    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98k6x\" (UniqueName: \"kubernetes.io/projected/8a641cc4-4ae2-4544-aef3-788a4a6606c1-kube-api-access-98k6x\") pod \"payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w\" (UID: \"8a641cc4-4ae2-4544-aef3-788a4a6606c1\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:56:27.303490 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.303458    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:56:27.343567 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.343531    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:56:27.446151 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.446123    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"]
Jul 29 18:56:27.449375 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:56:27.449332    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3af08e09_372f_4fc2_8578_b83397eecfad.slice/crio-7c610001fa14e664ddeab9b691872c107430c07f8b8fd42112599086548920cb WatchSource:0}: Error finding container 7c610001fa14e664ddeab9b691872c107430c07f8b8fd42112599086548920cb: Status 404 returned error can't find the container with id 7c610001fa14e664ddeab9b691872c107430c07f8b8fd42112599086548920cb
Jul 29 18:56:27.486833 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.486811    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"]
Jul 29 18:56:27.488737 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:56:27.488705    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a641cc4_4ae2_4544_aef3_788a4a6606c1.slice/crio-734df9e26c907180f1d95190a1c4e16b87207ffc98ca6221e890caeb1891f090 WatchSource:0}: Error finding container 734df9e26c907180f1d95190a1c4e16b87207ffc98ca6221e890caeb1891f090: Status 404 returned error can't find the container with id 734df9e26c907180f1d95190a1c4e16b87207ffc98ca6221e890caeb1891f090
Jul 29 18:56:27.729485 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.729390    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w" event={"ID":"8a641cc4-4ae2-4544-aef3-788a4a6606c1","Type":"ContainerStarted","Data":"861955e3a9f7cba354f26a74dfea5ac32fde6ac48e28541850b88989a6e9be6a"}
Jul 29 18:56:27.729485 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.729446    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w" event={"ID":"8a641cc4-4ae2-4544-aef3-788a4a6606c1","Type":"ContainerStarted","Data":"734df9e26c907180f1d95190a1c4e16b87207ffc98ca6221e890caeb1891f090"}
Jul 29 18:56:27.729732 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.729554    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:56:27.731078 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.731046    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g" event={"ID":"3af08e09-372f-4fc2-8578-b83397eecfad","Type":"ContainerStarted","Data":"faea262fbc521e0a16ff30f0fb1fa974f81f16456bb39e85062a41fb07bf3d79"}
Jul 29 18:56:27.731203 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.731092    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g" event={"ID":"3af08e09-372f-4fc2-8578-b83397eecfad","Type":"ContainerStarted","Data":"7c610001fa14e664ddeab9b691872c107430c07f8b8fd42112599086548920cb"}
Jul 29 18:56:27.731203 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.731134    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:56:27.746029 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.745959    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w" podStartSLOduration=0.745944823 podStartE2EDuration="745.944823ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:56:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:56:27.745279232 +0000 UTC m=+3857.253662844" watchObservedRunningTime="2026-07-29 18:56:27.745944823 +0000 UTC m=+3857.254328410"
Jul 29 18:56:27.763510 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:27.763463    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g" podStartSLOduration=1.7634450130000001 podStartE2EDuration="1.763445013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:56:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:56:27.76070753 +0000 UTC m=+3857.269091118" watchObservedRunningTime="2026-07-29 18:56:27.763445013 +0000 UTC m=+3857.271828605"
Jul 29 18:56:38.738537 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:38.738454    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:56:38.739108 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:38.738538    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:56:51.193706 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:51.193675    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:51.855866 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:51.855835    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:52.162970 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:52.162929    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:52.289552 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:52.289519    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:55.534772 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:55.534739    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:55.605282 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:55.605245    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:55.676063 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:55.676021    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:55.967146 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:55.967114    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:56.225706 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.225629    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:56.324088 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.324042    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:56:56.367268 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.367239    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"]
Jul 29 18:56:56.370777 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.370760    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 18:56:56.374372 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.374327    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-a10ee3-dockercfg-f2pmb\""
Jul 29 18:56:56.374581 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.374564    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-a10ee3\""
Jul 29 18:56:56.382950 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.382930    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"]
Jul 29 18:56:56.455819 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.455790    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb3453f0-7dc3-4882-8dc1-bf53fd167717-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg\" (UID: \"cb3453f0-7dc3-4882-8dc1-bf53fd167717\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 18:56:56.455978 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.455827    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fjv4\" (UniqueName: \"kubernetes.io/projected/cb3453f0-7dc3-4882-8dc1-bf53fd167717-kube-api-access-4fjv4\") pod \"payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg\" (UID: \"cb3453f0-7dc3-4882-8dc1-bf53fd167717\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 18:56:56.556336 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.556256    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb3453f0-7dc3-4882-8dc1-bf53fd167717-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg\" (UID: \"cb3453f0-7dc3-4882-8dc1-bf53fd167717\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 18:56:56.556336 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.556295    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fjv4\" (UniqueName: \"kubernetes.io/projected/cb3453f0-7dc3-4882-8dc1-bf53fd167717-kube-api-access-4fjv4\") pod \"payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg\" (UID: \"cb3453f0-7dc3-4882-8dc1-bf53fd167717\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 18:56:56.556988 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.556968    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb3453f0-7dc3-4882-8dc1-bf53fd167717-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg\" (UID: \"cb3453f0-7dc3-4882-8dc1-bf53fd167717\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 18:56:56.565119 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.565089    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fjv4\" (UniqueName: \"kubernetes.io/projected/cb3453f0-7dc3-4882-8dc1-bf53fd167717-kube-api-access-4fjv4\") pod \"payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg\" (UID: \"cb3453f0-7dc3-4882-8dc1-bf53fd167717\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 18:56:56.680710 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.680678    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 18:56:56.815927 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.815850    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"]
Jul 29 18:56:56.818949 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:56:56.818916    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3453f0_7dc3_4882_8dc1_bf53fd167717.slice/crio-2ae8d303c42c5780519d7c3fb76fc0c0398aada599c77e07c9e9c37a4a01df67 WatchSource:0}: Error finding container 2ae8d303c42c5780519d7c3fb76fc0c0398aada599c77e07c9e9c37a4a01df67: Status 404 returned error can't find the container with id 2ae8d303c42c5780519d7c3fb76fc0c0398aada599c77e07c9e9c37a4a01df67
Jul 29 18:56:56.859548 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:56.859521    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg" event={"ID":"cb3453f0-7dc3-4882-8dc1-bf53fd167717","Type":"ContainerStarted","Data":"2ae8d303c42c5780519d7c3fb76fc0c0398aada599c77e07c9e9c37a4a01df67"}
Jul 29 18:56:57.865246 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:57.865211    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg" event={"ID":"cb3453f0-7dc3-4882-8dc1-bf53fd167717","Type":"ContainerStarted","Data":"0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d"}
Jul 29 18:56:57.865747 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:57.865320    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 18:56:57.882510 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:56:57.882463    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg" podStartSLOduration=1.882449341 podStartE2EDuration="1.882449341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:56:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:56:57.880614052 +0000 UTC m=+3887.388997639" watchObservedRunningTime="2026-07-29 18:56:57.882449341 +0000 UTC m=+3887.390832928"
Jul 29 18:57:08.871996 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:08.871958    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 18:57:20.524744 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:20.524705    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:57:27.490263 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:27.490229    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg_cb3453f0-7dc3-4882-8dc1-bf53fd167717/payload-processing/0.log"
Jul 29 18:57:32.120627 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.120589    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"]
Jul 29 18:57:32.123870 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.123854    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.127518 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.127484    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-3140cc\"/\"kube-root-ca.crt\""
Jul 29 18:57:32.127670 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.127549    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-3140cc\"/\"ipp-route-3140cc-kserve-self-signed-certs\""
Jul 29 18:57:32.127670 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.127552    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-3140cc\"/\"default-dockercfg-fxmqk\""
Jul 29 18:57:32.127670 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.127565    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-3140cc\"/\"openshift-service-ca.crt\""
Jul 29 18:57:32.135529 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.135492    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"]
Jul 29 18:57:32.238244 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.238206    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:57:32.252354 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.252326    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bbt8\" (UniqueName: \"kubernetes.io/projected/95103e2e-7285-4b82-ab34-bf225f68e05f-kube-api-access-6bbt8\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.252540 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.252381    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-home\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.252540 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.252443    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95103e2e-7285-4b82-ab34-bf225f68e05f-tls-certs\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.252540 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.252489    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-model-cache\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.252540 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.252512    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-tmp-dir\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.252692 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.252550    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-dshm\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.354039 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.354003    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95103e2e-7285-4b82-ab34-bf225f68e05f-tls-certs\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.354246 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.354047    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-model-cache\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.354246 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.354069    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-tmp-dir\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.354246 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.354107    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-dshm\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.354246 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.354192    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bbt8\" (UniqueName: \"kubernetes.io/projected/95103e2e-7285-4b82-ab34-bf225f68e05f-kube-api-access-6bbt8\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.354246 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.354221    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-home\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.354568 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.354502    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-model-cache\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.354625 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.354557    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-tmp-dir\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.354625 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.354613    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-home\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.356391 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.356354    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-dshm\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.356573 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.356557    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95103e2e-7285-4b82-ab34-bf225f68e05f-tls-certs\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.362657 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.362635    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bbt8\" (UniqueName: \"kubernetes.io/projected/95103e2e-7285-4b82-ab34-bf225f68e05f-kube-api-access-6bbt8\") pod \"ipp-route-3140cc-kserve-67c99b8d6c-ccgwt\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") " pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.433558 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.433521    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:32.565096 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:32.565068    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"]
Jul 29 18:57:32.567520 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:57:32.567481    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95103e2e_7285_4b82_ab34_bf225f68e05f.slice/crio-3d138db46cab8bb0601918e7a84a085a54e1a697468ff7a913a0548e578f3e51 WatchSource:0}: Error finding container 3d138db46cab8bb0601918e7a84a085a54e1a697468ff7a913a0548e578f3e51: Status 404 returned error can't find the container with id 3d138db46cab8bb0601918e7a84a085a54e1a697468ff7a913a0548e578f3e51
Jul 29 18:57:33.018061 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:33.018023    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt" event={"ID":"95103e2e-7285-4b82-ab34-bf225f68e05f","Type":"ContainerStarted","Data":"6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab"}
Jul 29 18:57:33.018235 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:33.018069    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt" event={"ID":"95103e2e-7285-4b82-ab34-bf225f68e05f","Type":"ContainerStarted","Data":"3d138db46cab8bb0601918e7a84a085a54e1a697468ff7a913a0548e578f3e51"}
Jul 29 18:57:33.038766 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:33.038694    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt" podStartSLOduration=1.038674912 podStartE2EDuration="1.038674912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:57:33.035169989 +0000 UTC m=+3922.543553577" watchObservedRunningTime="2026-07-29 18:57:33.038674912 +0000 UTC m=+3922.547058502"
Jul 29 18:57:42.433989 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:42.433952    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:42.433989 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:42.433993    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:42.446556 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:42.446527    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:57:43.078123 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:57:43.078097    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:58:12.146352 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:12.146271    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w_8a641cc4-4ae2-4544-aef3-788a4a6606c1/payload-processing/0.log"
Jul 29 18:58:12.382951 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:12.382926    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg_cb3453f0-7dc3-4882-8dc1-bf53fd167717/payload-processing/0.log"
Jul 29 18:58:12.651230 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:12.651198    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:13.175303 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:13.175270    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:13.571201 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:13.571120    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:15.961209 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:15.961164    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:16.036472 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.036425    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:16.111865 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.111831    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:16.205043 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.205014    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:16.538835 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.538801    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:16.635597 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.635557    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"]
Jul 29 18:58:16.639124 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.639100    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:16.641555 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.641530    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-8c76b3\""
Jul 29 18:58:16.641672 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.641579    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-8c76b3-dockercfg-hrlx2\""
Jul 29 18:58:16.646591 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.646533    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"]
Jul 29 18:58:16.676930 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.676879    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"]
Jul 29 18:58:16.680057 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.680042    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:16.688942 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.688915    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"]
Jul 29 18:58:16.692440 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.692414    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:16.758506 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.758478    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e7b7981-f076-4a62-8529-75cfe0c3ead6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp\" (UID: \"8e7b7981-f076-4a62-8529-75cfe0c3ead6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:16.758654 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.758517    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfn7k\" (UniqueName: \"kubernetes.io/projected/8e7b7981-f076-4a62-8529-75cfe0c3ead6-kube-api-access-vfn7k\") pod \"payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp\" (UID: \"8e7b7981-f076-4a62-8529-75cfe0c3ead6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:16.758654 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.758552    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v98xx\" (UniqueName: \"kubernetes.io/projected/81249457-cde3-49cc-9b44-315e6a6824f9-kube-api-access-v98xx\") pod \"payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq\" (UID: \"81249457-cde3-49cc-9b44-315e6a6824f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:16.758654 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.758596    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81249457-cde3-49cc-9b44-315e6a6824f9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq\" (UID: \"81249457-cde3-49cc-9b44-315e6a6824f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:16.859383 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.859281    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e7b7981-f076-4a62-8529-75cfe0c3ead6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp\" (UID: \"8e7b7981-f076-4a62-8529-75cfe0c3ead6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:16.859383 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.859330    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfn7k\" (UniqueName: \"kubernetes.io/projected/8e7b7981-f076-4a62-8529-75cfe0c3ead6-kube-api-access-vfn7k\") pod \"payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp\" (UID: \"8e7b7981-f076-4a62-8529-75cfe0c3ead6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:16.859383 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.859382    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v98xx\" (UniqueName: \"kubernetes.io/projected/81249457-cde3-49cc-9b44-315e6a6824f9-kube-api-access-v98xx\") pod \"payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq\" (UID: \"81249457-cde3-49cc-9b44-315e6a6824f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:16.859617 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.859426    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81249457-cde3-49cc-9b44-315e6a6824f9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq\" (UID: \"81249457-cde3-49cc-9b44-315e6a6824f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:16.860029 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.860007    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e7b7981-f076-4a62-8529-75cfe0c3ead6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp\" (UID: \"8e7b7981-f076-4a62-8529-75cfe0c3ead6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:16.860108 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.860088    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81249457-cde3-49cc-9b44-315e6a6824f9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq\" (UID: \"81249457-cde3-49cc-9b44-315e6a6824f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:16.867484 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.867462    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfn7k\" (UniqueName: \"kubernetes.io/projected/8e7b7981-f076-4a62-8529-75cfe0c3ead6-kube-api-access-vfn7k\") pod \"payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp\" (UID: \"8e7b7981-f076-4a62-8529-75cfe0c3ead6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:16.867688 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.867668    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v98xx\" (UniqueName: \"kubernetes.io/projected/81249457-cde3-49cc-9b44-315e6a6824f9-kube-api-access-v98xx\") pod \"payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq\" (UID: \"81249457-cde3-49cc-9b44-315e6a6824f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:16.950485 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.950453    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:16.990826 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:16.990797    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:17.114717 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:17.114684    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"]
Jul 29 18:58:17.117984 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:58:17.117956    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7b7981_f076_4a62_8529_75cfe0c3ead6.slice/crio-310375478e3bf05d11043fd4b545eb257b71b99e78a38221e5d93a349e1e68eb WatchSource:0}: Error finding container 310375478e3bf05d11043fd4b545eb257b71b99e78a38221e5d93a349e1e68eb: Status 404 returned error can't find the container with id 310375478e3bf05d11043fd4b545eb257b71b99e78a38221e5d93a349e1e68eb
Jul 29 18:58:17.163907 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:17.163882    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"]
Jul 29 18:58:17.166005 ip-10-0-135-253 kubenswrapper[2556]: W0729 18:58:17.165985    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81249457_cde3_49cc_9b44_315e6a6824f9.slice/crio-e7c35ea2ed234cc2b5067422726095ce0afedd39b0008fae958cd583af5a638a WatchSource:0}: Error finding container e7c35ea2ed234cc2b5067422726095ce0afedd39b0008fae958cd583af5a638a: Status 404 returned error can't find the container with id e7c35ea2ed234cc2b5067422726095ce0afedd39b0008fae958cd583af5a638a
Jul 29 18:58:17.218417 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:17.218388    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp" event={"ID":"8e7b7981-f076-4a62-8529-75cfe0c3ead6","Type":"ContainerStarted","Data":"a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac"}
Jul 29 18:58:17.218730 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:17.218449    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp" event={"ID":"8e7b7981-f076-4a62-8529-75cfe0c3ead6","Type":"ContainerStarted","Data":"310375478e3bf05d11043fd4b545eb257b71b99e78a38221e5d93a349e1e68eb"}
Jul 29 18:58:17.218730 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:17.218480    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:17.219713 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:17.219687    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq" event={"ID":"81249457-cde3-49cc-9b44-315e6a6824f9","Type":"ContainerStarted","Data":"e7c35ea2ed234cc2b5067422726095ce0afedd39b0008fae958cd583af5a638a"}
Jul 29 18:58:17.251522 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:17.251469    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp" podStartSLOduration=1.251452288 podStartE2EDuration="1.251452288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:58:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:58:17.249194083 +0000 UTC m=+3966.757577671" watchObservedRunningTime="2026-07-29 18:58:17.251452288 +0000 UTC m=+3966.759835875"
Jul 29 18:58:18.226014 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:18.225975    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq" event={"ID":"81249457-cde3-49cc-9b44-315e6a6824f9","Type":"ContainerStarted","Data":"7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259"}
Jul 29 18:58:18.226433 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:18.226205    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:18.249882 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:18.249575    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq" podStartSLOduration=2.249533743 podStartE2EDuration="2.249533743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:58:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:58:18.242801595 +0000 UTC m=+3967.751185187" watchObservedRunningTime="2026-07-29 18:58:18.249533743 +0000 UTC m=+3967.757917336"
Jul 29 18:58:27.412599 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:27.412564    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:28.228557 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:28.228525    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:29.233508 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:29.233480    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:40.656657 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:40.656622    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:40.708536 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:40.708507    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:40.956761 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:40.956678    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:46.613859 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:46.613821    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:46.636896 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:46.636865    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"]
Jul 29 18:58:46.637142 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:46.637118    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq" podUID="81249457-cde3-49cc-9b44-315e6a6824f9" containerName="payload-processing" containerID="cri-o://7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259" gracePeriod=30
Jul 29 18:58:46.643206 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:46.643179    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"]
Jul 29 18:58:46.643740 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:46.643709    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp" podUID="8e7b7981-f076-4a62-8529-75cfe0c3ead6" containerName="payload-pre-processing" containerID="cri-o://a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac" gracePeriod=30
Jul 29 18:58:46.675974 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:46.675950    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:46.733143 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:46.733115    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:46.786804 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:46.786772    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:46.920980 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:46.920955    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:46.923048 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:46.923031    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:47.023828 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.023791    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e7b7981-f076-4a62-8529-75cfe0c3ead6-plugins-config-volume\") pod \"8e7b7981-f076-4a62-8529-75cfe0c3ead6\" (UID: \"8e7b7981-f076-4a62-8529-75cfe0c3ead6\") "
Jul 29 18:58:47.023828 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.023833    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfn7k\" (UniqueName: \"kubernetes.io/projected/8e7b7981-f076-4a62-8529-75cfe0c3ead6-kube-api-access-vfn7k\") pod \"8e7b7981-f076-4a62-8529-75cfe0c3ead6\" (UID: \"8e7b7981-f076-4a62-8529-75cfe0c3ead6\") "
Jul 29 18:58:47.024053 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.023859    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v98xx\" (UniqueName: \"kubernetes.io/projected/81249457-cde3-49cc-9b44-315e6a6824f9-kube-api-access-v98xx\") pod \"81249457-cde3-49cc-9b44-315e6a6824f9\" (UID: \"81249457-cde3-49cc-9b44-315e6a6824f9\") "
Jul 29 18:58:47.024053 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.023924    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81249457-cde3-49cc-9b44-315e6a6824f9-plugins-config-volume\") pod \"81249457-cde3-49cc-9b44-315e6a6824f9\" (UID: \"81249457-cde3-49cc-9b44-315e6a6824f9\") "
Jul 29 18:58:47.024338 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.024309    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e7b7981-f076-4a62-8529-75cfe0c3ead6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e7b7981-f076-4a62-8529-75cfe0c3ead6" (UID: "8e7b7981-f076-4a62-8529-75cfe0c3ead6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:58:47.024338 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.024332    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81249457-cde3-49cc-9b44-315e6a6824f9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "81249457-cde3-49cc-9b44-315e6a6824f9" (UID: "81249457-cde3-49cc-9b44-315e6a6824f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:58:47.026005 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.025965    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e7b7981-f076-4a62-8529-75cfe0c3ead6-kube-api-access-vfn7k" (OuterVolumeSpecName: "kube-api-access-vfn7k") pod "8e7b7981-f076-4a62-8529-75cfe0c3ead6" (UID: "8e7b7981-f076-4a62-8529-75cfe0c3ead6"). InnerVolumeSpecName "kube-api-access-vfn7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:58:47.026005 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.025985    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81249457-cde3-49cc-9b44-315e6a6824f9-kube-api-access-v98xx" (OuterVolumeSpecName: "kube-api-access-v98xx") pod "81249457-cde3-49cc-9b44-315e6a6824f9" (UID: "81249457-cde3-49cc-9b44-315e6a6824f9"). InnerVolumeSpecName "kube-api-access-v98xx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:58:47.125379 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.125339    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v98xx\" (UniqueName: \"kubernetes.io/projected/81249457-cde3-49cc-9b44-315e6a6824f9-kube-api-access-v98xx\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:58:47.125379 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.125382    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81249457-cde3-49cc-9b44-315e6a6824f9-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:58:47.125549 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.125393    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e7b7981-f076-4a62-8529-75cfe0c3ead6-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:58:47.125549 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.125403    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfn7k\" (UniqueName: \"kubernetes.io/projected/8e7b7981-f076-4a62-8529-75cfe0c3ead6-kube-api-access-vfn7k\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:58:47.132199 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.132167    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:47.366845 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.366738    2556 generic.go:356] "Generic (PLEG): container finished" podID="81249457-cde3-49cc-9b44-315e6a6824f9" containerID="7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259" exitCode=0
Jul 29 18:58:47.366845 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.366785    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"
Jul 29 18:58:47.366845 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.366820    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq" event={"ID":"81249457-cde3-49cc-9b44-315e6a6824f9","Type":"ContainerDied","Data":"7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259"}
Jul 29 18:58:47.367144 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.366870    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq" event={"ID":"81249457-cde3-49cc-9b44-315e6a6824f9","Type":"ContainerDied","Data":"e7c35ea2ed234cc2b5067422726095ce0afedd39b0008fae958cd583af5a638a"}
Jul 29 18:58:47.367144 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.366894    2556 scope.go:117] "RemoveContainer" containerID="7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259"
Jul 29 18:58:47.368182 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.368156    2556 generic.go:356] "Generic (PLEG): container finished" podID="8e7b7981-f076-4a62-8529-75cfe0c3ead6" containerID="a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac" exitCode=0
Jul 29 18:58:47.368302 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.368204    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp" event={"ID":"8e7b7981-f076-4a62-8529-75cfe0c3ead6","Type":"ContainerDied","Data":"a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac"}
Jul 29 18:58:47.368302 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.368220    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"
Jul 29 18:58:47.368302 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.368257    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp" event={"ID":"8e7b7981-f076-4a62-8529-75cfe0c3ead6","Type":"ContainerDied","Data":"310375478e3bf05d11043fd4b545eb257b71b99e78a38221e5d93a349e1e68eb"}
Jul 29 18:58:47.375673 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.375655    2556 scope.go:117] "RemoveContainer" containerID="7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259"
Jul 29 18:58:47.375956 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:58:47.375939    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259\": container with ID starting with 7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259 not found: ID does not exist" containerID="7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259"
Jul 29 18:58:47.376025 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.375963    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259"} err="failed to get container status \"7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259\": rpc error: code = NotFound desc = could not find container \"7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259\": container with ID starting with 7996f1af75be676d399922366f45485380429605a995def4396f637864eb8259 not found: ID does not exist"
Jul 29 18:58:47.376025 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.375978    2556 scope.go:117] "RemoveContainer" containerID="a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac"
Jul 29 18:58:47.383654 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.383619    2556 scope.go:117] "RemoveContainer" containerID="a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac"
Jul 29 18:58:47.384008 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:58:47.383950    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac\": container with ID starting with a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac not found: ID does not exist" containerID="a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac"
Jul 29 18:58:47.384008 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.383984    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac"} err="failed to get container status \"a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac\": rpc error: code = NotFound desc = could not find container \"a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac\": container with ID starting with a96e52c9578dc543336a65ac820c4e9ac4d32ca491cdf0ef46621addb3cac2ac not found: ID does not exist"
Jul 29 18:58:47.385693 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.385669    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"]
Jul 29 18:58:47.387799 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.387780    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8c76b3-75c954df6d-hwwlp"]
Jul 29 18:58:47.398301 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.398281    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"]
Jul 29 18:58:47.401915 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:47.401895    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8c76b3-6bc6457fc8-7w5mq"]
Jul 29 18:58:49.078919 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:49.078888    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81249457-cde3-49cc-9b44-315e6a6824f9" path="/var/lib/kubelet/pods/81249457-cde3-49cc-9b44-315e6a6824f9/volumes"
Jul 29 18:58:49.079303 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:49.079240    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e7b7981-f076-4a62-8529-75cfe0c3ead6" path="/var/lib/kubelet/pods/8e7b7981-f076-4a62-8529-75cfe0c3ead6/volumes"
Jul 29 18:58:52.790595 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:52.790565    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:52.890920 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:52.890886    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:53.049150 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:53.049065    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:58:53.270935 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:58:53.270904    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:59:12.266445 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.266410    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:59:12.296427 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.296400    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"]
Jul 29 18:59:12.296632 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.296608    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w" podUID="8a641cc4-4ae2-4544-aef3-788a4a6606c1" containerName="payload-processing" containerID="cri-o://861955e3a9f7cba354f26a74dfea5ac32fde6ac48e28541850b88989a6e9be6a" gracePeriod=30
Jul 29 18:59:12.304885 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.304858    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"]
Jul 29 18:59:12.305244 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.305217    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g" podUID="3af08e09-372f-4fc2-8578-b83397eecfad" containerName="payload-pre-processing" containerID="cri-o://faea262fbc521e0a16ff30f0fb1fa974f81f16456bb39e85062a41fb07bf3d79" gracePeriod=30
Jul 29 18:59:12.335854 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.335822    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:59:12.394952 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.394923    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:59:12.480163 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.480130    2556 generic.go:356] "Generic (PLEG): container finished" podID="3af08e09-372f-4fc2-8578-b83397eecfad" containerID="faea262fbc521e0a16ff30f0fb1fa974f81f16456bb39e85062a41fb07bf3d79" exitCode=0
Jul 29 18:59:12.480298 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.480201    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g" event={"ID":"3af08e09-372f-4fc2-8578-b83397eecfad","Type":"ContainerDied","Data":"faea262fbc521e0a16ff30f0fb1fa974f81f16456bb39e85062a41fb07bf3d79"}
Jul 29 18:59:12.481886 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.481853    2556 generic.go:356] "Generic (PLEG): container finished" podID="8a641cc4-4ae2-4544-aef3-788a4a6606c1" containerID="861955e3a9f7cba354f26a74dfea5ac32fde6ac48e28541850b88989a6e9be6a" exitCode=0
Jul 29 18:59:12.482003 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.481890    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w" event={"ID":"8a641cc4-4ae2-4544-aef3-788a4a6606c1","Type":"ContainerDied","Data":"861955e3a9f7cba354f26a74dfea5ac32fde6ac48e28541850b88989a6e9be6a"}
Jul 29 18:59:12.579040 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.579014    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:59:12.581668 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.581649    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:59:12.648889 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.648863    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98k6x\" (UniqueName: \"kubernetes.io/projected/8a641cc4-4ae2-4544-aef3-788a4a6606c1-kube-api-access-98k6x\") pod \"8a641cc4-4ae2-4544-aef3-788a4a6606c1\" (UID: \"8a641cc4-4ae2-4544-aef3-788a4a6606c1\") "
Jul 29 18:59:12.649051 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.648904    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn5hl\" (UniqueName: \"kubernetes.io/projected/3af08e09-372f-4fc2-8578-b83397eecfad-kube-api-access-rn5hl\") pod \"3af08e09-372f-4fc2-8578-b83397eecfad\" (UID: \"3af08e09-372f-4fc2-8578-b83397eecfad\") "
Jul 29 18:59:12.649051 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.648933    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3af08e09-372f-4fc2-8578-b83397eecfad-plugins-config-volume\") pod \"3af08e09-372f-4fc2-8578-b83397eecfad\" (UID: \"3af08e09-372f-4fc2-8578-b83397eecfad\") "
Jul 29 18:59:12.649051 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.648974    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a641cc4-4ae2-4544-aef3-788a4a6606c1-plugins-config-volume\") pod \"8a641cc4-4ae2-4544-aef3-788a4a6606c1\" (UID: \"8a641cc4-4ae2-4544-aef3-788a4a6606c1\") "
Jul 29 18:59:12.649457 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.649427    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3af08e09-372f-4fc2-8578-b83397eecfad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3af08e09-372f-4fc2-8578-b83397eecfad" (UID: "3af08e09-372f-4fc2-8578-b83397eecfad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:59:12.649457 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.649441    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a641cc4-4ae2-4544-aef3-788a4a6606c1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a641cc4-4ae2-4544-aef3-788a4a6606c1" (UID: "8a641cc4-4ae2-4544-aef3-788a4a6606c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:59:12.651092 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.651063    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a641cc4-4ae2-4544-aef3-788a4a6606c1-kube-api-access-98k6x" (OuterVolumeSpecName: "kube-api-access-98k6x") pod "8a641cc4-4ae2-4544-aef3-788a4a6606c1" (UID: "8a641cc4-4ae2-4544-aef3-788a4a6606c1"). InnerVolumeSpecName "kube-api-access-98k6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:59:12.651179 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.651130    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af08e09-372f-4fc2-8578-b83397eecfad-kube-api-access-rn5hl" (OuterVolumeSpecName: "kube-api-access-rn5hl") pod "3af08e09-372f-4fc2-8578-b83397eecfad" (UID: "3af08e09-372f-4fc2-8578-b83397eecfad"). InnerVolumeSpecName "kube-api-access-rn5hl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:59:12.750306 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.750272    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98k6x\" (UniqueName: \"kubernetes.io/projected/8a641cc4-4ae2-4544-aef3-788a4a6606c1-kube-api-access-98k6x\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:59:12.750306 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.750299    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn5hl\" (UniqueName: \"kubernetes.io/projected/3af08e09-372f-4fc2-8578-b83397eecfad-kube-api-access-rn5hl\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:59:12.750306 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.750308    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3af08e09-372f-4fc2-8578-b83397eecfad-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:59:12.750570 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:12.750317    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a641cc4-4ae2-4544-aef3-788a4a6606c1-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:59:13.337319 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.337246    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:59:13.391102 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.391076    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:59:13.487084 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.487055    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"
Jul 29 18:59:13.487240 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.487053    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w" event={"ID":"8a641cc4-4ae2-4544-aef3-788a4a6606c1","Type":"ContainerDied","Data":"734df9e26c907180f1d95190a1c4e16b87207ffc98ca6221e890caeb1891f090"}
Jul 29 18:59:13.487240 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.487213    2556 scope.go:117] "RemoveContainer" containerID="861955e3a9f7cba354f26a74dfea5ac32fde6ac48e28541850b88989a6e9be6a"
Jul 29 18:59:13.488306 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.488281    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g" event={"ID":"3af08e09-372f-4fc2-8578-b83397eecfad","Type":"ContainerDied","Data":"7c610001fa14e664ddeab9b691872c107430c07f8b8fd42112599086548920cb"}
Jul 29 18:59:13.488306 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.488298    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"
Jul 29 18:59:13.495616 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.495602    2556 scope.go:117] "RemoveContainer" containerID="faea262fbc521e0a16ff30f0fb1fa974f81f16456bb39e85062a41fb07bf3d79"
Jul 29 18:59:13.505417 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.505397    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"]
Jul 29 18:59:13.511500 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.511480    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3140cc-7c64895d9b-fcn8g"]
Jul 29 18:59:13.528408 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.528352    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"]
Jul 29 18:59:13.529738 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:13.529719    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3140cc-9c6f87cc8-qmp9w"]
Jul 29 18:59:15.078680 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:15.078635    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3af08e09-372f-4fc2-8578-b83397eecfad" path="/var/lib/kubelet/pods/3af08e09-372f-4fc2-8578-b83397eecfad/volumes"
Jul 29 18:59:15.079172 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:15.079142    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a641cc4-4ae2-4544-aef3-788a4a6606c1" path="/var/lib/kubelet/pods/8a641cc4-4ae2-4544-aef3-788a4a6606c1/volumes"
Jul 29 18:59:17.790964 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:17.790930    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"]
Jul 29 18:59:17.836681 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:17.836653    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:59:32.809713 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:32.809651    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt" podUID="95103e2e-7285-4b82-ab34-bf225f68e05f" containerName="main" containerID="cri-o://6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab" gracePeriod=45
Jul 29 18:59:33.067704 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.067647    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:59:33.125252 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.125209    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95103e2e-7285-4b82-ab34-bf225f68e05f-tls-certs\") pod \"95103e2e-7285-4b82-ab34-bf225f68e05f\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") "
Jul 29 18:59:33.125450 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.125275    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bbt8\" (UniqueName: \"kubernetes.io/projected/95103e2e-7285-4b82-ab34-bf225f68e05f-kube-api-access-6bbt8\") pod \"95103e2e-7285-4b82-ab34-bf225f68e05f\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") "
Jul 29 18:59:33.125450 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.125302    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-dshm\") pod \"95103e2e-7285-4b82-ab34-bf225f68e05f\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") "
Jul 29 18:59:33.125450 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.125321    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-tmp-dir\") pod \"95103e2e-7285-4b82-ab34-bf225f68e05f\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") "
Jul 29 18:59:33.125450 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.125356    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-model-cache\") pod \"95103e2e-7285-4b82-ab34-bf225f68e05f\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") "
Jul 29 18:59:33.125450 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.125413    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-home\") pod \"95103e2e-7285-4b82-ab34-bf225f68e05f\" (UID: \"95103e2e-7285-4b82-ab34-bf225f68e05f\") "
Jul 29 18:59:33.125752 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.125612    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "95103e2e-7285-4b82-ab34-bf225f68e05f" (UID: "95103e2e-7285-4b82-ab34-bf225f68e05f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:59:33.125818 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.125789    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-home" (OuterVolumeSpecName: "home") pod "95103e2e-7285-4b82-ab34-bf225f68e05f" (UID: "95103e2e-7285-4b82-ab34-bf225f68e05f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:59:33.125923 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.125895    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-model-cache" (OuterVolumeSpecName: "model-cache") pod "95103e2e-7285-4b82-ab34-bf225f68e05f" (UID: "95103e2e-7285-4b82-ab34-bf225f68e05f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:59:33.127635 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.127614    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-dshm" (OuterVolumeSpecName: "dshm") pod "95103e2e-7285-4b82-ab34-bf225f68e05f" (UID: "95103e2e-7285-4b82-ab34-bf225f68e05f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:59:33.127880 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.127859    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95103e2e-7285-4b82-ab34-bf225f68e05f-kube-api-access-6bbt8" (OuterVolumeSpecName: "kube-api-access-6bbt8") pod "95103e2e-7285-4b82-ab34-bf225f68e05f" (UID: "95103e2e-7285-4b82-ab34-bf225f68e05f"). InnerVolumeSpecName "kube-api-access-6bbt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:59:33.127939 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.127874    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95103e2e-7285-4b82-ab34-bf225f68e05f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "95103e2e-7285-4b82-ab34-bf225f68e05f" (UID: "95103e2e-7285-4b82-ab34-bf225f68e05f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:59:33.226553 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.226508    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-tmp-dir\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:59:33.226553 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.226548    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-model-cache\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:59:33.226553 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.226557    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-home\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:59:33.226553 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.226567    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95103e2e-7285-4b82-ab34-bf225f68e05f-tls-certs\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:59:33.226847 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.226576    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bbt8\" (UniqueName: \"kubernetes.io/projected/95103e2e-7285-4b82-ab34-bf225f68e05f-kube-api-access-6bbt8\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:59:33.226847 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.226584    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95103e2e-7285-4b82-ab34-bf225f68e05f-dshm\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 18:59:33.575395 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.575296    2556 generic.go:356] "Generic (PLEG): container finished" podID="95103e2e-7285-4b82-ab34-bf225f68e05f" containerID="6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab" exitCode=0
Jul 29 18:59:33.575395 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.575337    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt" event={"ID":"95103e2e-7285-4b82-ab34-bf225f68e05f","Type":"ContainerDied","Data":"6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab"}
Jul 29 18:59:33.575395 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.575337    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"
Jul 29 18:59:33.575700 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.575406    2556 scope.go:117] "RemoveContainer" containerID="6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab"
Jul 29 18:59:33.575700 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.575391    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt" event={"ID":"95103e2e-7285-4b82-ab34-bf225f68e05f","Type":"ContainerDied","Data":"3d138db46cab8bb0601918e7a84a085a54e1a697468ff7a913a0548e578f3e51"}
Jul 29 18:59:33.583964 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.583947    2556 scope.go:117] "RemoveContainer" containerID="6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab"
Jul 29 18:59:33.584208 ip-10-0-135-253 kubenswrapper[2556]: E0729 18:59:33.584189    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab\": container with ID starting with 6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab not found: ID does not exist" containerID="6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab"
Jul 29 18:59:33.584253 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.584218    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab"} err="failed to get container status \"6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab\": rpc error: code = NotFound desc = could not find container \"6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab\": container with ID starting with 6263c8f53d5a37fe62c12728ce04b5f1992c546c9b4d6e8462dd8643a895ffab not found: ID does not exist"
Jul 29 18:59:33.596332 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.596305    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"]
Jul 29 18:59:33.600300 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:33.600277    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-3140cc/ipp-route-3140cc-kserve-67c99b8d6c-ccgwt"]
Jul 29 18:59:35.083021 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:35.082977    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95103e2e-7285-4b82-ab34-bf225f68e05f" path="/var/lib/kubelet/pods/95103e2e-7285-4b82-ab34-bf225f68e05f/volumes"
Jul 29 18:59:54.230546 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:54.230516    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:59:54.642199 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:54.642173    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 18:59:54.870235 ip-10-0-135-253 kubenswrapper[2556]: I0729 18:59:54.870192    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:00.060566 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:00.060533    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:22.751974 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:22.751936    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:22.820952 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:22.820919    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:23.067021 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:23.066923    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:23.286547 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:23.286508    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:33.824244 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:33.824209    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:33.854510 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:33.854472    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"]
Jul 29 19:00:33.854752 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:33.854726    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg" podUID="cb3453f0-7dc3-4882-8dc1-bf53fd167717" containerName="payload-processing" containerID="cri-o://0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d" gracePeriod=30
Jul 29 19:00:33.888226 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:33.888193    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:33.991192 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:33.991160    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:34.096794 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.096767    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:34.108804 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.108785    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 19:00:34.155622 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.155594    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb3453f0-7dc3-4882-8dc1-bf53fd167717-plugins-config-volume\") pod \"cb3453f0-7dc3-4882-8dc1-bf53fd167717\" (UID: \"cb3453f0-7dc3-4882-8dc1-bf53fd167717\") "
Jul 29 19:00:34.155757 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.155675    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fjv4\" (UniqueName: \"kubernetes.io/projected/cb3453f0-7dc3-4882-8dc1-bf53fd167717-kube-api-access-4fjv4\") pod \"cb3453f0-7dc3-4882-8dc1-bf53fd167717\" (UID: \"cb3453f0-7dc3-4882-8dc1-bf53fd167717\") "
Jul 29 19:00:34.156054 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.156020    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb3453f0-7dc3-4882-8dc1-bf53fd167717-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cb3453f0-7dc3-4882-8dc1-bf53fd167717" (UID: "cb3453f0-7dc3-4882-8dc1-bf53fd167717"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:00:34.157699 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.157675    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb3453f0-7dc3-4882-8dc1-bf53fd167717-kube-api-access-4fjv4" (OuterVolumeSpecName: "kube-api-access-4fjv4") pod "cb3453f0-7dc3-4882-8dc1-bf53fd167717" (UID: "cb3453f0-7dc3-4882-8dc1-bf53fd167717"). InnerVolumeSpecName "kube-api-access-4fjv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:00:34.257159 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.257127    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb3453f0-7dc3-4882-8dc1-bf53fd167717-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 19:00:34.257159 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.257158    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fjv4\" (UniqueName: \"kubernetes.io/projected/cb3453f0-7dc3-4882-8dc1-bf53fd167717-kube-api-access-4fjv4\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 19:00:34.367857 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.367777    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:34.841208 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.841172    2556 generic.go:356] "Generic (PLEG): container finished" podID="cb3453f0-7dc3-4882-8dc1-bf53fd167717" containerID="0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d" exitCode=0
Jul 29 19:00:34.841698 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.841218    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"
Jul 29 19:00:34.841698 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.841226    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg" event={"ID":"cb3453f0-7dc3-4882-8dc1-bf53fd167717","Type":"ContainerDied","Data":"0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d"}
Jul 29 19:00:34.841698 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.841274    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg" event={"ID":"cb3453f0-7dc3-4882-8dc1-bf53fd167717","Type":"ContainerDied","Data":"2ae8d303c42c5780519d7c3fb76fc0c0398aada599c77e07c9e9c37a4a01df67"}
Jul 29 19:00:34.841698 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.841298    2556 scope.go:117] "RemoveContainer" containerID="0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d"
Jul 29 19:00:34.852954 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.852934    2556 scope.go:117] "RemoveContainer" containerID="0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d"
Jul 29 19:00:34.853233 ip-10-0-135-253 kubenswrapper[2556]: E0729 19:00:34.853208    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d\": container with ID starting with 0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d not found: ID does not exist" containerID="0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d"
Jul 29 19:00:34.853307 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.853241    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d"} err="failed to get container status \"0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d\": rpc error: code = NotFound desc = could not find container \"0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d\": container with ID starting with 0164f2fc4cd8e5016ba0d1ee15a0f1fb48988279d75c4c79e9ee24a2ab91080d not found: ID does not exist"
Jul 29 19:00:34.863962 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.863939    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"]
Jul 29 19:00:34.868862 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:34.868839    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a10ee3-56c5bcd78-d4cjg"]
Jul 29 19:00:35.083684 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:35.083639    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb3453f0-7dc3-4882-8dc1-bf53fd167717" path="/var/lib/kubelet/pods/cb3453f0-7dc3-4882-8dc1-bf53fd167717/volumes"
Jul 29 19:00:40.115298 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:40.115266    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:40.184073 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:40.184041    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:40.362412 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:40.362383    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:40.585163 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:40.585133    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:51.221176 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.221145    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:51.240319 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.240290    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"]
Jul 29 19:00:51.240529 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.240510    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht" podUID="b8a934b6-fb6a-49fa-b336-f05c60c74642" containerName="payload-processing" containerID="cri-o://6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d" gracePeriod=30
Jul 29 19:00:51.247192 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.247162    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"]
Jul 29 19:00:51.247456 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.247387    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk" podUID="0665d60e-fe13-4f4a-9b52-c88699847c96" containerName="payload-pre-processing" containerID="cri-o://671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668" gracePeriod=30
Jul 29 19:00:51.320536 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.320511    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:51.509382 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.509342    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 19:00:51.511977 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.511957    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 19:00:51.599575 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.599541    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0665d60e-fe13-4f4a-9b52-c88699847c96-plugins-config-volume\") pod \"0665d60e-fe13-4f4a-9b52-c88699847c96\" (UID: \"0665d60e-fe13-4f4a-9b52-c88699847c96\") "
Jul 29 19:00:51.599743 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.599584    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbkhd\" (UniqueName: \"kubernetes.io/projected/0665d60e-fe13-4f4a-9b52-c88699847c96-kube-api-access-wbkhd\") pod \"0665d60e-fe13-4f4a-9b52-c88699847c96\" (UID: \"0665d60e-fe13-4f4a-9b52-c88699847c96\") "
Jul 29 19:00:51.599743 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.599611    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjdgk\" (UniqueName: \"kubernetes.io/projected/b8a934b6-fb6a-49fa-b336-f05c60c74642-kube-api-access-hjdgk\") pod \"b8a934b6-fb6a-49fa-b336-f05c60c74642\" (UID: \"b8a934b6-fb6a-49fa-b336-f05c60c74642\") "
Jul 29 19:00:51.599743 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.599642    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8a934b6-fb6a-49fa-b336-f05c60c74642-plugins-config-volume\") pod \"b8a934b6-fb6a-49fa-b336-f05c60c74642\" (UID: \"b8a934b6-fb6a-49fa-b336-f05c60c74642\") "
Jul 29 19:00:51.600085 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.600052    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0665d60e-fe13-4f4a-9b52-c88699847c96-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0665d60e-fe13-4f4a-9b52-c88699847c96" (UID: "0665d60e-fe13-4f4a-9b52-c88699847c96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:00:51.600189 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.600059    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8a934b6-fb6a-49fa-b336-f05c60c74642-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8a934b6-fb6a-49fa-b336-f05c60c74642" (UID: "b8a934b6-fb6a-49fa-b336-f05c60c74642"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:00:51.601666 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.601638    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8a934b6-fb6a-49fa-b336-f05c60c74642-kube-api-access-hjdgk" (OuterVolumeSpecName: "kube-api-access-hjdgk") pod "b8a934b6-fb6a-49fa-b336-f05c60c74642" (UID: "b8a934b6-fb6a-49fa-b336-f05c60c74642"). InnerVolumeSpecName "kube-api-access-hjdgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:00:51.601778 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.601705    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0665d60e-fe13-4f4a-9b52-c88699847c96-kube-api-access-wbkhd" (OuterVolumeSpecName: "kube-api-access-wbkhd") pod "0665d60e-fe13-4f4a-9b52-c88699847c96" (UID: "0665d60e-fe13-4f4a-9b52-c88699847c96"). InnerVolumeSpecName "kube-api-access-wbkhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:00:51.700980 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.700954    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjdgk\" (UniqueName: \"kubernetes.io/projected/b8a934b6-fb6a-49fa-b336-f05c60c74642-kube-api-access-hjdgk\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 19:00:51.700980 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.700978    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8a934b6-fb6a-49fa-b336-f05c60c74642-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 19:00:51.700980 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.700989    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0665d60e-fe13-4f4a-9b52-c88699847c96-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 19:00:51.701198 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.700997    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbkhd\" (UniqueName: \"kubernetes.io/projected/0665d60e-fe13-4f4a-9b52-c88699847c96-kube-api-access-wbkhd\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 19:00:51.914767 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.914736    2556 generic.go:356] "Generic (PLEG): container finished" podID="0665d60e-fe13-4f4a-9b52-c88699847c96" containerID="671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668" exitCode=0
Jul 29 19:00:51.914947 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.914777    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"
Jul 29 19:00:51.914947 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.914821    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk" event={"ID":"0665d60e-fe13-4f4a-9b52-c88699847c96","Type":"ContainerDied","Data":"671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668"}
Jul 29 19:00:51.914947 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.914884    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk" event={"ID":"0665d60e-fe13-4f4a-9b52-c88699847c96","Type":"ContainerDied","Data":"e804406c1649790b0acc25d39f26dd1f53161dfee666ed30c541403e27ff2c19"}
Jul 29 19:00:51.914947 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.914920    2556 scope.go:117] "RemoveContainer" containerID="671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668"
Jul 29 19:00:51.916126 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.916107    2556 generic.go:356] "Generic (PLEG): container finished" podID="b8a934b6-fb6a-49fa-b336-f05c60c74642" containerID="6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d" exitCode=0
Jul 29 19:00:51.916214 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.916148    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"
Jul 29 19:00:51.916214 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.916152    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht" event={"ID":"b8a934b6-fb6a-49fa-b336-f05c60c74642","Type":"ContainerDied","Data":"6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d"}
Jul 29 19:00:51.916214 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.916185    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht" event={"ID":"b8a934b6-fb6a-49fa-b336-f05c60c74642","Type":"ContainerDied","Data":"c97c29fbf57c284e13fb281c841e5e60f7ef7ad055df9fdc43cbe38a6c50d1e1"}
Jul 29 19:00:51.924081 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.924062    2556 scope.go:117] "RemoveContainer" containerID="671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668"
Jul 29 19:00:51.924381 ip-10-0-135-253 kubenswrapper[2556]: E0729 19:00:51.924345    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668\": container with ID starting with 671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668 not found: ID does not exist" containerID="671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668"
Jul 29 19:00:51.924478 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.924391    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668"} err="failed to get container status \"671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668\": rpc error: code = NotFound desc = could not find container \"671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668\": container with ID starting with 671a2ffc418293e1293f36bdfda333fdb968894cfc514390148c94c8a66d5668 not found: ID does not exist"
Jul 29 19:00:51.924478 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.924408    2556 scope.go:117] "RemoveContainer" containerID="6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d"
Jul 29 19:00:51.931774 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.931757    2556 scope.go:117] "RemoveContainer" containerID="6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d"
Jul 29 19:00:51.932017 ip-10-0-135-253 kubenswrapper[2556]: E0729 19:00:51.931996    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d\": container with ID starting with 6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d not found: ID does not exist" containerID="6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d"
Jul 29 19:00:51.932066 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.932026    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d"} err="failed to get container status \"6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d\": rpc error: code = NotFound desc = could not find container \"6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d\": container with ID starting with 6718e74fd36cdff6366509ec1d61548e6676d178b3e8c4a1bbb500754597db9d not found: ID does not exist"
Jul 29 19:00:51.939913 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.939890    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"]
Jul 29 19:00:51.943955 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.943934    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6e9884-75574bf484-54knk"]
Jul 29 19:00:51.953962 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.953941    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"]
Jul 29 19:00:51.957828 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:51.957806    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6e9884-7f59f8b6db-wdjht"]
Jul 29 19:00:53.079494 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:53.079465    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0665d60e-fe13-4f4a-9b52-c88699847c96" path="/var/lib/kubelet/pods/0665d60e-fe13-4f4a-9b52-c88699847c96/volumes"
Jul 29 19:00:53.079858 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:53.079811    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8a934b6-fb6a-49fa-b336-f05c60c74642" path="/var/lib/kubelet/pods/b8a934b6-fb6a-49fa-b336-f05c60c74642/volumes"
Jul 29 19:00:53.501583 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:53.501555    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:53.606578 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:53.606551    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:57.488720 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:57.488682    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:57.590048 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:57.590013    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:57.751974 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:57.751896    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:00:57.969195 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:00:57.969160    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:01:08.502266 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.502185    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:01:08.526720 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.526689    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"]
Jul 29 19:01:08.526995 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.526968    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j" podUID="6ae94ec0-f0bb-4d3b-a75d-0b91491881b9" containerName="payload-processing" containerID="cri-o://d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964" gracePeriod=30
Jul 29 19:01:08.534469 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.534437    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"]
Jul 29 19:01:08.535010 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.534636    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5" podUID="515d7540-f685-42f7-815a-85d53aa3969d" containerName="payload-pre-processing" containerID="cri-o://4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5" gracePeriod=30
Jul 29 19:01:08.797201 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.797179    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 19:01:08.799582 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.799566    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 19:01:08.855986 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.855957    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cr8vw\" (UniqueName: \"kubernetes.io/projected/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-kube-api-access-cr8vw\") pod \"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9\" (UID: \"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9\") "
Jul 29 19:01:08.856138 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.856009    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4b8mr\" (UniqueName: \"kubernetes.io/projected/515d7540-f685-42f7-815a-85d53aa3969d-kube-api-access-4b8mr\") pod \"515d7540-f685-42f7-815a-85d53aa3969d\" (UID: \"515d7540-f685-42f7-815a-85d53aa3969d\") "
Jul 29 19:01:08.856138 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.856033    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-plugins-config-volume\") pod \"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9\" (UID: \"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9\") "
Jul 29 19:01:08.856138 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.856061    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/515d7540-f685-42f7-815a-85d53aa3969d-plugins-config-volume\") pod \"515d7540-f685-42f7-815a-85d53aa3969d\" (UID: \"515d7540-f685-42f7-815a-85d53aa3969d\") "
Jul 29 19:01:08.856457 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.856424    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ae94ec0-f0bb-4d3b-a75d-0b91491881b9" (UID: "6ae94ec0-f0bb-4d3b-a75d-0b91491881b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:01:08.856457 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.856431    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/515d7540-f685-42f7-815a-85d53aa3969d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "515d7540-f685-42f7-815a-85d53aa3969d" (UID: "515d7540-f685-42f7-815a-85d53aa3969d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:01:08.858035 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.858009    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/515d7540-f685-42f7-815a-85d53aa3969d-kube-api-access-4b8mr" (OuterVolumeSpecName: "kube-api-access-4b8mr") pod "515d7540-f685-42f7-815a-85d53aa3969d" (UID: "515d7540-f685-42f7-815a-85d53aa3969d"). InnerVolumeSpecName "kube-api-access-4b8mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:01:08.858112 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.858065    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-kube-api-access-cr8vw" (OuterVolumeSpecName: "kube-api-access-cr8vw") pod "6ae94ec0-f0bb-4d3b-a75d-0b91491881b9" (UID: "6ae94ec0-f0bb-4d3b-a75d-0b91491881b9"). InnerVolumeSpecName "kube-api-access-cr8vw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:01:08.956988 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.956949    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr8vw\" (UniqueName: \"kubernetes.io/projected/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-kube-api-access-cr8vw\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 19:01:08.956988 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.956981    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b8mr\" (UniqueName: \"kubernetes.io/projected/515d7540-f685-42f7-815a-85d53aa3969d-kube-api-access-4b8mr\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 19:01:08.956988 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.956990    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 19:01:08.956988 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.957001    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/515d7540-f685-42f7-815a-85d53aa3969d-plugins-config-volume\") on node \"ip-10-0-135-253.ec2.internal\" DevicePath \"\""
Jul 29 19:01:08.995710 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.995674    2556 generic.go:356] "Generic (PLEG): container finished" podID="6ae94ec0-f0bb-4d3b-a75d-0b91491881b9" containerID="d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964" exitCode=0
Jul 29 19:01:08.995868 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.995726    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"
Jul 29 19:01:08.995868 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.995754    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j" event={"ID":"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9","Type":"ContainerDied","Data":"d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964"}
Jul 29 19:01:08.995868 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.995814    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j" event={"ID":"6ae94ec0-f0bb-4d3b-a75d-0b91491881b9","Type":"ContainerDied","Data":"3e51b8dcd71d7797d2c6c5234a4dc68cc11c31813b93b383a41adf01859d17b6"}
Jul 29 19:01:08.995868 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.995844    2556 scope.go:117] "RemoveContainer" containerID="d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964"
Jul 29 19:01:08.997074 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.997052    2556 generic.go:356] "Generic (PLEG): container finished" podID="515d7540-f685-42f7-815a-85d53aa3969d" containerID="4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5" exitCode=0
Jul 29 19:01:08.997153 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.997091    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"
Jul 29 19:01:08.997206 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.997092    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5" event={"ID":"515d7540-f685-42f7-815a-85d53aa3969d","Type":"ContainerDied","Data":"4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5"}
Jul 29 19:01:08.997263 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:08.997229    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5" event={"ID":"515d7540-f685-42f7-815a-85d53aa3969d","Type":"ContainerDied","Data":"605b8f20fb205f82380a24939347b63561e3f8ffd243730477a83b347864b2b0"}
Jul 29 19:01:09.004803 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:09.004715    2556 scope.go:117] "RemoveContainer" containerID="d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964"
Jul 29 19:01:09.005036 ip-10-0-135-253 kubenswrapper[2556]: E0729 19:01:09.005012    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964\": container with ID starting with d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964 not found: ID does not exist" containerID="d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964"
Jul 29 19:01:09.005109 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:09.005050    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964"} err="failed to get container status \"d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964\": rpc error: code = NotFound desc = could not find container \"d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964\": container with ID starting with d7c1bb0611575de9636f529221ff761747e84ed19db7b81fbf97747e151d0964 not found: ID does not exist"
Jul 29 19:01:09.005109 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:09.005074    2556 scope.go:117] "RemoveContainer" containerID="4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5"
Jul 29 19:01:09.012415 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:09.012399    2556 scope.go:117] "RemoveContainer" containerID="4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5"
Jul 29 19:01:09.012660 ip-10-0-135-253 kubenswrapper[2556]: E0729 19:01:09.012642    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5\": container with ID starting with 4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5 not found: ID does not exist" containerID="4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5"
Jul 29 19:01:09.012719 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:09.012664    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5"} err="failed to get container status \"4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5\": rpc error: code = NotFound desc = could not find container \"4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5\": container with ID starting with 4a52cd1cfcb4110a6f38bbd6bf993aa2173c5c54067a2ec510b206bbd284a5d5 not found: ID does not exist"
Jul 29 19:01:09.024415 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:09.019728    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"]
Jul 29 19:01:09.026232 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:09.026208    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-750f0d-67644b6bcb-8d7n5"]
Jul 29 19:01:09.036944 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:09.036925    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"]
Jul 29 19:01:09.038590 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:09.038572    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-750f0d-966f7bf98-dpj6j"]
Jul 29 19:01:09.078934 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:09.078909    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="515d7540-f685-42f7-815a-85d53aa3969d" path="/var/lib/kubelet/pods/515d7540-f685-42f7-815a-85d53aa3969d/volumes"
Jul 29 19:01:09.079237 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:09.079225    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ae94ec0-f0bb-4d3b-a75d-0b91491881b9" path="/var/lib/kubelet/pods/6ae94ec0-f0bb-4d3b-a75d-0b91491881b9/volumes"
Jul 29 19:01:10.342392 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:10.342351    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:01:14.832203 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:14.832175    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:01:15.010339 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:15.010301    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:01:15.128009 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:15.127984    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:01:15.343110 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:15.343075    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:01:15.754859 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:15.754817    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78c979c496-s7kr6_a83195f9-3b92-4940-be8e-1a13a58f5464/authorino/0.log"
Jul 29 19:01:21.666324 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:21.666294    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-h7psx_c775513b-de7a-40fb-8d99-2dcdac5526e4/manager/0.log"
Jul 29 19:01:21.672725 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:21.672705    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-h7psx_c775513b-de7a-40fb-8d99-2dcdac5526e4/copy-manifests/0.log"
Jul 29 19:01:23.840206 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:23.840171    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78c979c496-s7kr6_a83195f9-3b92-4940-be8e-1a13a58f5464/authorino/0.log"
Jul 29 19:01:24.195254 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:24.195222    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mf6sk_0533fe48-2d39-42f5-af54-ad93e6de2f8a/kuadrant-console-plugin/0.log"
Jul 29 19:01:25.222545 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:25.222508    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869f956dcc-7hq9m_d3825976-667d-40ba-95ec-48b5b62ece45/kube-auth-proxy/0.log"
Jul 29 19:01:26.149831 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:26.149791    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-2w7gg_5c1e378d-dd3b-41cc-a7aa-2032dd6101e7/main/0.log"
Jul 29 19:01:26.485727 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:26.485644    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4ld9j_5517c982-2206-4211-934d-51908a3dc879/main/0.log"
Jul 29 19:01:26.707855 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:26.707826    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-kwxkr_66a6528d-514b-4995-849a-ca898f933b92/main/0.log"
Jul 29 19:01:32.142891 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:32.142856    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:01:32.241786 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:32.241749    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log" limit=1024
Jul 29 19:01:35.174976 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:35.174944    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5qf55_9a962e36-54ac-40e3-8c51-c317c81f2133/global-pull-secret-syncer/0.log"
Jul 29 19:01:35.302663 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:35.302636    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2sqk8_026c3a54-f3f1-4f0f-88d6-681bd039200d/konnectivity-agent/0.log"
Jul 29 19:01:35.461475 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:35.461397    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-253.ec2.internal_e9a47ae04fab76d00c2743727b114e18/haproxy/0.log"
Jul 29 19:01:41.292589 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:41.292558    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78c979c496-s7kr6_a83195f9-3b92-4940-be8e-1a13a58f5464/authorino/0.log"
Jul 29 19:01:41.386666 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:41.386639    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mf6sk_0533fe48-2d39-42f5-af54-ad93e6de2f8a/kuadrant-console-plugin/0.log"
Jul 29 19:01:43.776076 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:43.776042    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nxhrj_af725a92-e993-462d-a9e6-99f1298b08c0/node-exporter/0.log"
Jul 29 19:01:43.799352 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:43.799329    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nxhrj_af725a92-e993-462d-a9e6-99f1298b08c0/kube-rbac-proxy/0.log"
Jul 29 19:01:43.823232 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:43.823209    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nxhrj_af725a92-e993-462d-a9e6-99f1298b08c0/init-textfile/0.log"
Jul 29 19:01:45.603257 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603219    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"]
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603518    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="515d7540-f685-42f7-815a-85d53aa3969d" containerName="payload-pre-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603530    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="515d7540-f685-42f7-815a-85d53aa3969d" containerName="payload-pre-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603541    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3af08e09-372f-4fc2-8578-b83397eecfad" containerName="payload-pre-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603547    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3af08e09-372f-4fc2-8578-b83397eecfad" containerName="payload-pre-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603555    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95103e2e-7285-4b82-ab34-bf225f68e05f" containerName="main"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603561    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="95103e2e-7285-4b82-ab34-bf225f68e05f" containerName="main"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603570    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0665d60e-fe13-4f4a-9b52-c88699847c96" containerName="payload-pre-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603576    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0665d60e-fe13-4f4a-9b52-c88699847c96" containerName="payload-pre-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603582    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81249457-cde3-49cc-9b44-315e6a6824f9" containerName="payload-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603588    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="81249457-cde3-49cc-9b44-315e6a6824f9" containerName="payload-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603595    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a641cc4-4ae2-4544-aef3-788a4a6606c1" containerName="payload-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603600    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a641cc4-4ae2-4544-aef3-788a4a6606c1" containerName="payload-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603606    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8a934b6-fb6a-49fa-b336-f05c60c74642" containerName="payload-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603611    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8a934b6-fb6a-49fa-b336-f05c60c74642" containerName="payload-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603616    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb3453f0-7dc3-4882-8dc1-bf53fd167717" containerName="payload-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603621    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb3453f0-7dc3-4882-8dc1-bf53fd167717" containerName="payload-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603627    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ae94ec0-f0bb-4d3b-a75d-0b91491881b9" containerName="payload-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603632    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ae94ec0-f0bb-4d3b-a75d-0b91491881b9" containerName="payload-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603637    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e7b7981-f076-4a62-8529-75cfe0c3ead6" containerName="payload-pre-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603641    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e7b7981-f076-4a62-8529-75cfe0c3ead6" containerName="payload-pre-processing"
Jul 29 19:01:45.603675 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603684    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="515d7540-f685-42f7-815a-85d53aa3969d" containerName="payload-pre-processing"
Jul 29 19:01:45.604429 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603692    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e7b7981-f076-4a62-8529-75cfe0c3ead6" containerName="payload-pre-processing"
Jul 29 19:01:45.604429 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603698    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0665d60e-fe13-4f4a-9b52-c88699847c96" containerName="payload-pre-processing"
Jul 29 19:01:45.604429 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603703    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="81249457-cde3-49cc-9b44-315e6a6824f9" containerName="payload-processing"
Jul 29 19:01:45.604429 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603709    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a641cc4-4ae2-4544-aef3-788a4a6606c1" containerName="payload-processing"
Jul 29 19:01:45.604429 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603713    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ae94ec0-f0bb-4d3b-a75d-0b91491881b9" containerName="payload-processing"
Jul 29 19:01:45.604429 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603718    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8a934b6-fb6a-49fa-b336-f05c60c74642" containerName="payload-processing"
Jul 29 19:01:45.604429 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603726    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3af08e09-372f-4fc2-8578-b83397eecfad" containerName="payload-pre-processing"
Jul 29 19:01:45.604429 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603732    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb3453f0-7dc3-4882-8dc1-bf53fd167717" containerName="payload-processing"
Jul 29 19:01:45.604429 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.603739    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="95103e2e-7285-4b82-ab34-bf225f68e05f" containerName="main"
Jul 29 19:01:45.606808 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.606789    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.609284 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.609260    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5k8jh\"/\"kube-root-ca.crt\""
Jul 29 19:01:45.609430 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.609260    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5k8jh\"/\"openshift-service-ca.crt\""
Jul 29 19:01:45.609430 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.609317    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5k8jh\"/\"default-dockercfg-9zp9k\""
Jul 29 19:01:45.613713 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.613689    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"]
Jul 29 19:01:45.760859 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.760823    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kw4n\" (UniqueName: \"kubernetes.io/projected/80d03180-0899-47c6-bed3-f7211f5d02a1-kube-api-access-4kw4n\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.760859 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.760870    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-proc\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.761085 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.760891    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-lib-modules\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.761085 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.760996    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-sys\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.761085 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.761025    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-podres\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.861535 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.861443    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-sys\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.861535 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.861477    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-podres\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.861535 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.861520    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kw4n\" (UniqueName: \"kubernetes.io/projected/80d03180-0899-47c6-bed3-f7211f5d02a1-kube-api-access-4kw4n\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.861816 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.861542    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-proc\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.861816 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.861559    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-lib-modules\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.861816 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.861590    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-sys\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.861816 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.861622    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-podres\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.861816 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.861644    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-proc\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.861816 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.861705    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80d03180-0899-47c6-bed3-f7211f5d02a1-lib-modules\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.869694 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.869669    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kw4n\" (UniqueName: \"kubernetes.io/projected/80d03180-0899-47c6-bed3-f7211f5d02a1-kube-api-access-4kw4n\") pod \"perf-node-gather-daemonset-94lq2\" (UID: \"80d03180-0899-47c6-bed3-f7211f5d02a1\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:45.917905 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:45.917871    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:46.044066 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:46.044040    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"]
Jul 29 19:01:46.046667 ip-10-0-135-253 kubenswrapper[2556]: W0729 19:01:46.046634    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod80d03180_0899_47c6_bed3_f7211f5d02a1.slice/crio-a244fe57d283968488a0afce7e5d1fdb41d20b6d7338501f003d650ac59ed395 WatchSource:0}: Error finding container a244fe57d283968488a0afce7e5d1fdb41d20b6d7338501f003d650ac59ed395: Status 404 returned error can't find the container with id a244fe57d283968488a0afce7e5d1fdb41d20b6d7338501f003d650ac59ed395
Jul 29 19:01:46.145732 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:46.145707    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2" event={"ID":"80d03180-0899-47c6-bed3-f7211f5d02a1","Type":"ContainerStarted","Data":"1725a6a051296051802dc20268b0342a435582111edf67c3fffcbf523be5da3f"}
Jul 29 19:01:46.145929 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:46.145759    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2" event={"ID":"80d03180-0899-47c6-bed3-f7211f5d02a1","Type":"ContainerStarted","Data":"a244fe57d283968488a0afce7e5d1fdb41d20b6d7338501f003d650ac59ed395"}
Jul 29 19:01:46.145929 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:46.145780    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:46.160167 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:46.160122    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2" podStartSLOduration=1.160107009 podStartE2EDuration="1.160107009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:01:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:01:46.159637415 +0000 UTC m=+4175.668021002" watchObservedRunningTime="2026-07-29 19:01:46.160107009 +0000 UTC m=+4175.668490598"
Jul 29 19:01:48.285198 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:48.285168    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8v82s_dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e/dns/0.log"
Jul 29 19:01:48.310233 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:48.310206    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8v82s_dcc6927f-2ddf-4b0a-a52f-24fbfc2aad7e/kube-rbac-proxy/0.log"
Jul 29 19:01:48.476404 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:48.476355    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jf97c_3927a342-2b8b-47bd-bbb3-f78bbce3802a/dns-node-resolver/0.log"
Jul 29 19:01:48.994619 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:48.994586    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-84f857b7cd-rfs4d_b62d4027-7580-4b8a-a699-f90c6b318f59/registry/0.log"
Jul 29 19:01:49.015882 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:49.015858    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cttw7_eb4927fd-bc0a-4316-b43f-fb9144018eab/node-ca/0.log"
Jul 29 19:01:50.296567 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:50.296537    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869f956dcc-7hq9m_d3825976-667d-40ba-95ec-48b5b62ece45/kube-auth-proxy/0.log"
Jul 29 19:01:51.369961 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:51.369934    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jhjsc_37daa72f-68a5-462f-87f4-cbe6981c77a0/serve-healthcheck-canary/0.log"
Jul 29 19:01:51.951205 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:51.951175    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hzmb7_26658a95-087c-47ea-9a71-bb1cc1f85801/kube-rbac-proxy/0.log"
Jul 29 19:01:51.975480 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:51.975446    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hzmb7_26658a95-087c-47ea-9a71-bb1cc1f85801/exporter/0.log"
Jul 29 19:01:51.998332 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:51.998301    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hzmb7_26658a95-087c-47ea-9a71-bb1cc1f85801/extractor/0.log"
Jul 29 19:01:52.158981 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:52.158955    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-94lq2"
Jul 29 19:01:55.427497 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:55.427467    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-h7psx_c775513b-de7a-40fb-8d99-2dcdac5526e4/manager/0.log"
Jul 29 19:01:55.446061 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:55.446037    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-h7psx_c775513b-de7a-40fb-8d99-2dcdac5526e4/copy-manifests/0.log"
Jul 29 19:01:56.779378 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:56.779334    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7ff9fc5cb5-2rzm8_27c763e9-bdd5-4f9d-b384-a1c359cad1e8/manager/0.log"
Jul 29 19:01:56.829894 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:56.829867    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-qrw2c_acf028f5-4881-447d-b887-59e5e1b4d3d7/openshift-lws-operator/0.log"
Jul 29 19:01:56.966743 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:56.966713    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869f956dcc-7hq9m_d3825976-667d-40ba-95ec-48b5b62ece45/kube-auth-proxy/0.log"
Jul 29 19:01:58.742003 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:01:58.741959    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-869f956dcc-7hq9m_d3825976-667d-40ba-95ec-48b5b62ece45/kube-auth-proxy/0.log"
Jul 29 19:02:01.745235 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:02:01.745204    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-8zkjz_a98b8bc6-2242-4c67-abca-75e6b55c5d1f/migrator/0.log"
Jul 29 19:02:01.764833 ip-10-0-135-253 kubenswrapper[2556]: I0729 19:02:01.764812    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-8zkjz_a98b8bc6-2242-4c67-abca-75e6b55c5d1f/graceful-termination/0.log"