Jul 28 19:46:58.712564 ip-10-0-134-138 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 19:46:59.084888 ip-10-0-134-138 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:46:59.084888 ip-10-0-134-138 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 19:46:59.084888 ip-10-0-134-138 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:46:59.084888 ip-10-0-134-138 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 19:46:59.084888 ip-10-0-134-138 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:46:59.085641 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.084959    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 19:46:59.088220 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088204    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:46:59.088220 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088219    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:46:59.088220 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088223    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088226    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088230    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088233    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088236    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088239    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088241    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088244    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088247    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088250    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088253    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088256    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088259    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088262    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088264    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088267    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088269    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088272    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088274    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088289    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:46:59.088365 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088292    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088295    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088297    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088300    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088303    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088305    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088308    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088311    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088314    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088317    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088319    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088322    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088325    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088327    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088330    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088333    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088336    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088338    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088341    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088343    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:46:59.089050 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088346    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088349    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088351    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088354    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088356    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088359    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088361    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088364    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088368    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088372    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088375    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088378    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088381    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088384    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088387    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088389    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088392    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088394    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088397    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:46:59.089807 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088399    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088402    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088406    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088409    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088412    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088415    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088418    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088421    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088424    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088427    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088431    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088433    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088436    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088438    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088441    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088443    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088446    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088449    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088451    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088454    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:46:59.090492 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088456    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088458    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088461    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088463    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088467    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088471    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088475    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088477    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088481    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088483    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088486    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088488    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088491    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088493    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088496    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088498    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088501    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.088504    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089150    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089156    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:46:59.091187 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089159    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089162    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089164    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089168    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089170    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089173    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089176    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089179    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089181    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089184    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089188    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089192    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089195    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089198    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089201    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089204    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089207    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089210    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089213    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:46:59.091905 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089215    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089218    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089221    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089223    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089226    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089228    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089231    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089233    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089235    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089238    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089241    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089245    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089248    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089250    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089253    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089255    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089258    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089264    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089268    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089270    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:46:59.092656 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089273    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089289    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089292    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089295    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089298    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089300    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089303    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089305    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089309    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089311    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089314    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089316    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089319    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089322    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089324    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089327    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089330    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089333    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089336    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:46:59.093375 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089339    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089342    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089344    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089348    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089351    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089353    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089356    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089358    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089361    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089363    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089366    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089369    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089372    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089374    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089377    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089379    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089382    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089385    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089388    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089391    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:46:59.094059 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089393    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089395    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089398    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089400    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089403    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089406    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089408    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089411    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089413    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089416    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089418    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089421    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089423    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089425    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089428    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089430    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089433    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089436    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089438    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089844    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:46:59.094782 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089857    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089861    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089864    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089868    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089871    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089874    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089877    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089879    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089882    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089885    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089887    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089890    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089892    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089895    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089897    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089900    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089902    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089905    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089907    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089909    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:46:59.095524 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089912    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089915    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089918    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089920    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089923    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089925    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089928    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089932    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089935    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089937    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089939    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089942    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089945    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089947    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089949    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089952    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089955    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089957    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089959    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089962    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:46:59.096232 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089964    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089967    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089969    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089971    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089974    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089976    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089979    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089981    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089984    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089986    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089988    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089991    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089993    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089995    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.089998    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090000    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090003    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090005    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090010    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090013    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:46:59.096964 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090015    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090018    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090021    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090023    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090026    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090028    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090031    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090034    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090036    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090039    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090041    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090044    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090046    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090049    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090051    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090054    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090058    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090061    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090064    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090067    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:46:59.097717 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090069    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090072    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090075    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090077    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090079    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090083    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090088    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090091    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090093    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090096    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090100    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090103    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090106    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090109    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090113    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090116    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090118    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090121    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090180    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090186    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 19:46:59.098443 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090192    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090196    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090200    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090203    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090207    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090211    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090214    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090217    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090220    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090223    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090226    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090229    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090232    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090235    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090237    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090240    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090244    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090247    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090250    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090253    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090256    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090260    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090263    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 19:46:59.099145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090267    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090270    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090273    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090288    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090292    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090295    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090298    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090302    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090305    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090307    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090310    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090313    2569 flags.go:64] FLAG: --enable-server="true"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090317    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090321    2569 flags.go:64] FLAG: --event-burst="100"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090323    2569 flags.go:64] FLAG: --event-qps="50"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090326    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090329    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090332    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090336    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090338    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090341    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090344    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090347    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090350    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090352    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 19:46:59.099997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090355    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090358    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090361    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090365    2569 flags.go:64] FLAG: --feature-gates=""
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090368    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090371    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090375    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090378    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090382    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090385    2569 flags.go:64] FLAG: --help="false"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090387    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090390    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090393    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090396    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090399    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090403    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090405    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090408    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090411    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090414    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090417    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090420    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090423    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090426    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 19:46:59.100914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090428    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090431    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090434    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090437    2569 flags.go:64] FLAG: --lock-file=""
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090439    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090442    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090445    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090450    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090452    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090455    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090458    2569 flags.go:64] FLAG: --logging-format="text"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090462    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090465    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090468    2569 flags.go:64] FLAG: --manifest-url=""
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090472    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090476    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090479    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090483    2569 flags.go:64] FLAG: --max-pods="110"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090486    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090489    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090492    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090494    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090498    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090500    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090504    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 19:46:59.101792 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090511    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090514    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090517    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090520    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090523    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090532    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090535    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090538    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090541    2569 flags.go:64] FLAG: --port="10250"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090544    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090547    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d19f8bd7406a4982"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090550    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090552    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090555    2569 flags.go:64] FLAG: --register-node="true"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090558    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090566    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090569    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090571    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090574    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090578    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090581    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090586    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090588    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090591    2569 flags.go:64] FLAG: --runonce="false"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090594    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 19:46:59.102701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090597    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090600    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090603    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090605    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090608    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090611    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090614    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090617    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090620    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090622    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090625    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090628    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090631    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090634    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090639    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090641    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090644    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090647    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090650    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090653    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090656    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090658    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090661    2569 flags.go:64] FLAG: --v="2"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090665    2569 flags.go:64] FLAG: --version="false"
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090668    2569 flags.go:64] FLAG: --vmodule=""
Jul 28 19:46:59.103591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090671    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.090675    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090777    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090783    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090786    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090789    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090792    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090795    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090798    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090800    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090803    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090806    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090808    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090811    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090814    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090817    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090819    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090822    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090825    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:46:59.104533 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090827    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090830    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090833    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090836    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090838    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090841    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090844    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090848    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090851    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090853    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090856    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090858    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090861    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090863    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090866    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090869    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090873    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090875    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090878    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:46:59.105226 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090881    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090883    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090886    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090888    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090891    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090893    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090896    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090898    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090901    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090903    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090906    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090908    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090910    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090913    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090915    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090918    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090921    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090923    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090926    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090928    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:46:59.105958 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090931    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090933    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090936    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090938    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090941    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090943    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090946    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090949    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090951    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090955    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090957    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090960    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090962    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090965    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090968    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090970    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090972    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090975    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090978    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:46:59.106677 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090981    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090985    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090987    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090992    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090995    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.090998    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091001    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091003    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091006    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091009    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091011    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091014    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091016    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091019    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091022    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091024    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091027    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091029    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091033    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091036    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:46:59.107352 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091038    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091041    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091046    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.091048    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.091054    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.097219    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.097233    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.097251    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.097257    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097329    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097335    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097338    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097342    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097345    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097348    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:46:59.108048 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097351    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097354    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097357    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097360    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097363    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097366    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097369    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097372    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097375    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097377    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097380    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097382    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097385    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097387    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097390    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097392    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097395    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097398    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097400    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097403    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:46:59.108668 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097406    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097408    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097411    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097414    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097417    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097420    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097422    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097425    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097427    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097430    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097432    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097435    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097437    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097440    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097442    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097445    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097447    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097450    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097452    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:46:59.109490 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097456    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097460    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097463    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097466    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097468    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097471    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097474    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097476    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097479    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097481    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097484    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097486    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097489    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097492    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097495    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097497    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097500    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097503    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097505    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097508    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:46:59.110016 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097510    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097512    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097515    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097517    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097520    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097522    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097525    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097527    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097530    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097533    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097535    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097538    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097540    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097543    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097545    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097548    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097550    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097552    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097555    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:46:59.110566 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097557    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097561    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097565    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097567    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097571    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097574    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097577    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097580    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097584    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097586    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097589    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097592    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097594    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097597    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:46:59.111029 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097599    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.097604    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097705    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097710    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097714    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097718    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097722    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097724    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097728    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097731    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097733    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097736    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097739    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097742    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097745    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:46:59.111398 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097748    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097750    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097752    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097755    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097758    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097760    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097762    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097765    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097767    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097770    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097773    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097775    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097778    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097780    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097783    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097785    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097788    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097791    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097794    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097797    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:46:59.111768 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097799    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097802    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097804    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097806    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097809    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097812    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097816    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097819    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097822    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097824    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097827    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097830    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097832    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097835    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097837    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097840    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097842    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097845    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097847    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:46:59.112457 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097858    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097861    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097863    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097866    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097869    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097871    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097874    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097876    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097879    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097882    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097884    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097887    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097890    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097892    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097894    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097897    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097899    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097902    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097905    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:46:59.112960 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097907    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097910    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097912    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097915    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097917    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097920    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097922    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097925    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097927    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097931    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097934    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097937    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097939    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097942    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097944    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097950    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097952    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097955    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097958    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:46:59.113493 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097960    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097963    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097965    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097967    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097970    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097973    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097975    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097978    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:46:59.097981    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.097985    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.098086    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.099948    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.101210    2569 server.go:1019] "Starting client certificate rotation"
Jul 28 19:46:59.113963 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.101302    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:46:59.114305 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.101337    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:46:59.114305 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.105149    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:46:59.114305 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.109001    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:46:59.118870 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.118855    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 28 19:46:59.120748 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.120736    2569 log.go:25] "Validated CRI v1 image API"
Jul 28 19:46:59.122787 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.122772    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 19:46:59.127014 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.126995    2569 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c5289451-a68f-4e8b-b02c-c5d75738213f:/dev/nvme0n1p4 ff524c25-0f68-433d-9177-c014a168212f:/dev/nvme0n1p3]
Jul 28 19:46:59.127099 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.127013    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 19:46:59.130447 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.130422    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:46:59.132596 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.132492    2569 manager.go:216] Machine: {Timestamp:2026-07-28 19:46:59.130484452 +0000 UTC m=+0.311001497 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098832 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f981de712e2eb5b68ad3fd3449320 SystemUUID:ec2f981d-e712-e2eb-5b68-ad3fd3449320 BootID:6fe25bc2-c454-492b-8958-9e29cddc535d Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e4:ef:01:cd:d3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e4:ef:01:cd:d3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:e3:08:ef:89:3a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 19:46:59.132596 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.132590    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 19:46:59.132719 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.132654    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 19:46:59.134135 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.134111    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 19:46:59.134324 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.134140    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-138.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 19:46:59.134766 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.134755    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 19:46:59.134830 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.134773    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 19:46:59.134830 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.134783    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:46:59.135422 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.135411    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:46:59.136450 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.136439    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 19:46:59.140230 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.140217    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:46:59.140394 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.140383    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 19:46:59.140456 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.140429    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 19:46:59.140832 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.140821    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 19:46:59.140892 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.140841    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 19:46:59.140892 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.140854    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 28 19:46:59.140892 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.140865    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 19:46:59.141227 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.141214    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:46:59.141320 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.141237    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:46:59.143545 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.143529    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 19:46:59.145062 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.145043    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 19:46:59.145144 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.145074    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 19:46:59.146439 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146422    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 19:46:59.146439 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146442    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 19:46:59.146547 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146449    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 19:46:59.146547 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146456    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 19:46:59.146547 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146465    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 19:46:59.146547 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146471    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 19:46:59.146547 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146479    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 19:46:59.146547 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146489    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 19:46:59.146547 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146496    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 19:46:59.146547 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146502    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 19:46:59.146547 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146511    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 19:46:59.146547 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.146517    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 19:46:59.147417 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.147404    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 19:46:59.147459 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.147425    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 19:46:59.150720 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.150702    2569 server.go:1268] "Started kubelet"
Jul 28 19:46:59.150831 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.150812    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zv7r2"
Jul 28 19:46:59.150888 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.150867    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 19:46:59.150975 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.150936    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 19:46:59.151027 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.150992    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 19:46:59.151485 ip-10-0-134-138 systemd[1]: Started Kubernetes Kubelet.
Jul 28 19:46:59.152236 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.152208    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 19:46:59.152330 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.152242    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-138.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 19:46:59.152386 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.152343    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-138.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 19:46:59.152706 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.152690    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 19:46:59.152807 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.152793    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 19:46:59.155735 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.155718    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 19:46:59.156160 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.155271    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-138.ec2.internal.18c68bcdfe5734bd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-138.ec2.internal,UID:ip-10-0-134-138.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-138.ec2.internal,},FirstTimestamp:2026-07-28 19:46:59.150664893 +0000 UTC m=+0.331181944,LastTimestamp:2026-07-28 19:46:59.150664893 +0000 UTC m=+0.331181944,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-138.ec2.internal,}"
Jul 28 19:46:59.156294 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.156269    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zv7r2"
Jul 28 19:46:59.158102 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.158083    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 19:46:59.158102 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.158090    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 19:46:59.158242 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.158214    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:46:59.158783 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.158764    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 19:46:59.158783 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.158765    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 19:46:59.158928 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.158792    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 19:46:59.158928 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.158878    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 19:46:59.158928 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.158888    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 19:46:59.159072 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.159048    2569 factory.go:55] Registering systemd factory
Jul 28 19:46:59.159120 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.159073    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 28 19:46:59.159313 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.159297    2569 factory.go:153] Registering CRI-O factory
Jul 28 19:46:59.159313 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.159312    2569 factory.go:223] Registration of the crio container factory successfully
Jul 28 19:46:59.159462 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.159354    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 19:46:59.159462 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.159377    2569 factory.go:103] Registering Raw factory
Jul 28 19:46:59.159462 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.159393    2569 manager.go:1196] Started watching for new ooms in manager
Jul 28 19:46:59.160556 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.160541    2569 manager.go:349] Starting recovery of all containers
Jul 28 19:46:59.165458 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.165435    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:46:59.169173 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.169149    2569 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-138.ec2.internal\" not found" node="ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.169723 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.169706    2569 manager.go:354] Recovery completed
Jul 28 19:46:59.174623 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.174531    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:46:59.176102 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.176083    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:46:59.176174 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.176114    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:46:59.176174 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.176129    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:46:59.176613 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.176598    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 19:46:59.176613 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.176611    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 19:46:59.176713 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.176630    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:46:59.176713 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.176646    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:46:59.177344 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.177328    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:46:59.177443 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.177357    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:46:59.177443 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.177367    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:46:59.179018 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.179006    2569 policy_none.go:49] "None policy: Start"
Jul 28 19:46:59.179069 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.179023    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 19:46:59.179069 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.179033    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 19:46:59.180878 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.180868    2569 policy_none.go:47] "Start"
Jul 28 19:46:59.216933 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.216920    2569 manager.go:337] "Starting Device Plugin manager"
Jul 28 19:46:59.216996 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.216954    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 19:46:59.216996 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.216965    2569 server.go:87] "Starting device plugin registration server"
Jul 28 19:46:59.217179 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.217166    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 19:46:59.217234 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.217180    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 19:46:59.217309 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.217292    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 19:46:59.217418 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.217391    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 19:46:59.217418 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.217415    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 19:46:59.217878 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.217861    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 19:46:59.217954 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.217901    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:46:59.317554 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.317535    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:46:59.319381 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.318225    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:46:59.319381 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.318251    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:46:59.319381 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.318259    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:46:59.319381 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.318298    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.326774 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.326760    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.326847 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.326777    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-138.ec2.internal\": node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:46:59.339596 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.339551    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:46:59.358288 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.358248    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 19:46:59.359370 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.359353    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 19:46:59.359436 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.359399    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 19:46:59.359436 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.359427    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 19:46:59.359529 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.359465    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 19:46:59.362203 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.362182    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:46:59.440315 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.440292    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:46:59.459591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.459571    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"]
Jul 28 19:46:59.459666 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.459655    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:46:59.460462 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.460447    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:46:59.460523 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.460482    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:46:59.460523 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.460493    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:46:59.462814 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.462801    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:46:59.462951 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.462935    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.462996 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.462970    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:46:59.463488 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.463471    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:46:59.463552 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.463499    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:46:59.463552 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.463510    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:46:59.463656 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.463508    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:46:59.463691 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.463675    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:46:59.463691 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.463689    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:46:59.465975 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.465956    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.466067 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.465982    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:46:59.466649 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.466636    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:46:59.466727 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.466660    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:46:59.466727 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.466672    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:46:59.488030 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.488009    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-138.ec2.internal\" not found" node="ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.492775 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.492759    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-138.ec2.internal\" not found" node="ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.540853 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.540829    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:46:59.560054 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.560029    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1450dc71fe2617f4b2c6cd952b92073e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"1450dc71fe2617f4b2c6cd952b92073e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.560153 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.560056    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f39b57f429370ddf00b5bf9b9f3d71ea-config\") pod \"kube-apiserver-proxy-ip-10-0-134-138.ec2.internal\" (UID: \"f39b57f429370ddf00b5bf9b9f3d71ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.560153 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.560072    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1450dc71fe2617f4b2c6cd952b92073e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"1450dc71fe2617f4b2c6cd952b92073e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.642971 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.642915    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:46:59.660604 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.660585    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1450dc71fe2617f4b2c6cd952b92073e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"1450dc71fe2617f4b2c6cd952b92073e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.660658 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.660614    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1450dc71fe2617f4b2c6cd952b92073e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"1450dc71fe2617f4b2c6cd952b92073e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.660658 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.660634    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f39b57f429370ddf00b5bf9b9f3d71ea-config\") pod \"kube-apiserver-proxy-ip-10-0-134-138.ec2.internal\" (UID: \"f39b57f429370ddf00b5bf9b9f3d71ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.660766 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.660667    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f39b57f429370ddf00b5bf9b9f3d71ea-config\") pod \"kube-apiserver-proxy-ip-10-0-134-138.ec2.internal\" (UID: \"f39b57f429370ddf00b5bf9b9f3d71ea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.660766 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.660683    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1450dc71fe2617f4b2c6cd952b92073e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"1450dc71fe2617f4b2c6cd952b92073e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.660766 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.660734    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1450dc71fe2617f4b2c6cd952b92073e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal\" (UID: \"1450dc71fe2617f4b2c6cd952b92073e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.749534 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.749512    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:46:59.791134 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.791102    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.796708 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:46:59.796011    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 28 19:46:59.849819 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.849789    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:46:59.955445 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:46:59.955419    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:47:00.057266 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:00.057238    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:47:00.101289 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.101260    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 19:47:00.101728 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.101401    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 19:47:00.101728 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.101422    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 19:47:00.158274 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:00.158248    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:47:00.162592 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.161610    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 19:41:59 +0000 UTC" deadline="2028-03-25 05:39:35.278569445 +0000 UTC"
Jul 28 19:47:00.162592 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.161646    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14529h52m35.116928393s"
Jul 28 19:47:00.258638 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:00.258580    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:47:00.323909 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.323888    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:47:00.358856 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:00.358825    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:47:00.411938 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:00.411905    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39b57f429370ddf00b5bf9b9f3d71ea.slice/crio-72f2494068b5958681c99fb8ae34098e760b65d8bb24d1b10407ac683afcceae WatchSource:0}: Error finding container 72f2494068b5958681c99fb8ae34098e760b65d8bb24d1b10407ac683afcceae: Status 404 returned error can't find the container with id 72f2494068b5958681c99fb8ae34098e760b65d8bb24d1b10407ac683afcceae
Jul 28 19:47:00.412146 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:00.412115    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1450dc71fe2617f4b2c6cd952b92073e.slice/crio-9b7deb250b45a4d9c0917c467f88d2981713705f15beb31b5fc949b82ab1604d WatchSource:0}: Error finding container 9b7deb250b45a4d9c0917c467f88d2981713705f15beb31b5fc949b82ab1604d: Status 404 returned error can't find the container with id 9b7deb250b45a4d9c0917c467f88d2981713705f15beb31b5fc949b82ab1604d
Jul 28 19:47:00.415900 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.415877    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:47:00.459804 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:00.459782    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:47:00.559885 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:00.559827    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:47:00.660338 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:00.660293    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-138.ec2.internal\" not found"
Jul 28 19:47:00.691368 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.691345    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:47:00.763237 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.763211    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal"
Jul 28 19:47:00.769816 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.769794    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:47:00.770858 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.770842    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal"
Jul 28 19:47:00.779682 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:00.779664    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:47:01.142673 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.142640    2569 apiserver.go:52] "Watching apiserver"
Jul 28 19:47:01.151392 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.151361    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 19:47:01.152866 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.152822    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal","openshift-cluster-node-tuning-operator/tuned-q7wzh","openshift-image-registry/node-ca-swxbg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal","openshift-multus/multus-additional-cni-plugins-2hqwp","openshift-ovn-kubernetes/ovnkube-node-5m4m9","kube-system/global-pull-secret-syncer-hcb5b","kube-system/konnectivity-agent-gfhzh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q","openshift-dns/node-resolver-2m75c","openshift-multus/multus-qjrvq","openshift-multus/network-metrics-daemon-vwvtz","openshift-network-diagnostics/network-check-target-qcl5s","openshift-network-operator/iptables-alerter-kd2kj"]
Jul 28 19:47:01.155521 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.155485    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2m75c"
Jul 28 19:47:01.157493 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.157322    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 19:47:01.157493 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.157359    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jh7gg\""
Jul 28 19:47:01.157493 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.157385    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 19:47:01.157831 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.157806    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:01.157948 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.157885    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:01.158975 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.158670    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 19:47:01.160382 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.160355    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-swxbg"
Jul 28 19:47:01.162141 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.162119    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 19:47:01.162540 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.162520    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 19:47:01.162540 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.162532    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 19:47:01.162824 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.162805    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kd2kj"
Jul 28 19:47:01.162921 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.162891    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8xhmz\""
Jul 28 19:47:01.165312 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.165165    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ptpmw\""
Jul 28 19:47:01.165312 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.165197    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:47:01.165497 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.165475    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.165581 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.165562    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 19:47:01.165646 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.165574    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 19:47:01.167171 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.167148    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:47:01.167287 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.167265    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gtzzl\""
Jul 28 19:47:01.167529 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.167514    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 19:47:01.167619 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.167597    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 19:47:01.167848 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.167829    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 19:47:01.167927 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.167830    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 19:47:01.167984 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.167836    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 19:47:01.168972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.168951    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.169968 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.169944    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aee74d7d-4f10-43b5-b805-90cd335b79fd-tmp-dir\") pod \"node-resolver-2m75c\" (UID: \"aee74d7d-4f10-43b5-b805-90cd335b79fd\") " pod="openshift-dns/node-resolver-2m75c"
Jul 28 19:47:01.170069 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.169987    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxm2w\" (UniqueName: \"kubernetes.io/projected/aee74d7d-4f10-43b5-b805-90cd335b79fd-kube-api-access-kxm2w\") pod \"node-resolver-2m75c\" (UID: \"aee74d7d-4f10-43b5-b805-90cd335b79fd\") " pod="openshift-dns/node-resolver-2m75c"
Jul 28 19:47:01.170069 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.170028    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ad78daf-e4fe-4ef6-b833-308bd3211ea3-host\") pod \"node-ca-swxbg\" (UID: \"9ad78daf-e4fe-4ef6-b833-308bd3211ea3\") " pod="openshift-image-registry/node-ca-swxbg"
Jul 28 19:47:01.170069 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.170054    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ad78daf-e4fe-4ef6-b833-308bd3211ea3-serviceca\") pod \"node-ca-swxbg\" (UID: \"9ad78daf-e4fe-4ef6-b833-308bd3211ea3\") " pod="openshift-image-registry/node-ca-swxbg"
Jul 28 19:47:01.170238 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.170103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4fq7\" (UniqueName: \"kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7\") pod \"network-check-target-qcl5s\" (UID: \"8a92e62e-b631-4596-8efc-b4e65ff8e290\") " pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:01.170238 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.170157    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wcr4\" (UniqueName: \"kubernetes.io/projected/9ad78daf-e4fe-4ef6-b833-308bd3211ea3-kube-api-access-9wcr4\") pod \"node-ca-swxbg\" (UID: \"9ad78daf-e4fe-4ef6-b833-308bd3211ea3\") " pod="openshift-image-registry/node-ca-swxbg"
Jul 28 19:47:01.170238 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.170188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aee74d7d-4f10-43b5-b805-90cd335b79fd-hosts-file\") pod \"node-resolver-2m75c\" (UID: \"aee74d7d-4f10-43b5-b805-90cd335b79fd\") " pod="openshift-dns/node-resolver-2m75c"
Jul 28 19:47:01.171492 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.170965    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 19:47:01.171492 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.171305    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 19:47:01.171646 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.171560    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-n2qqq\""
Jul 28 19:47:01.171646 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.171608    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 19:47:01.172565 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.171875    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 19:47:01.172565 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.171942    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.172565 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.172443    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 19:47:01.172766 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.172690    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 19:47:01.173824 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.173802    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 19:47:01.174314 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.174292    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 19:47:01.174586 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.174554    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bbjrs\""
Jul 28 19:47:01.174727 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.174660    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 19:47:01.174940 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.174923    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:01.175043 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.174996    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:01.177557 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.177541    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.179239 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.179178    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:47:01.179580 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.179560    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kj677\""
Jul 28 19:47:01.179580 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.179572    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 19:47:01.180174 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.180154    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:01.182164 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.181879    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 19:47:01.182164 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.181925    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 19:47:01.182164 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.181971    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dmqfh\""
Jul 28 19:47:01.183347 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.183255    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.185963 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.185701    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 19:47:01.185963 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.185725    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fjrsb\""
Jul 28 19:47:01.185963 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.185922    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:01.186183 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.186010    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:01.193880 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.193518    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mqtsg"
Jul 28 19:47:01.201960 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.201918    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mqtsg"
Jul 28 19:47:01.260126 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.260022    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 19:47:01.265313 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.265124    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:47:01.270587 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270562    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxm2w\" (UniqueName: \"kubernetes.io/projected/aee74d7d-4f10-43b5-b805-90cd335b79fd-kube-api-access-kxm2w\") pod \"node-resolver-2m75c\" (UID: \"aee74d7d-4f10-43b5-b805-90cd335b79fd\") " pod="openshift-dns/node-resolver-2m75c"
Jul 28 19:47:01.270716 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270607    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s48k7\" (UniqueName: \"kubernetes.io/projected/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-kube-api-access-s48k7\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.270716 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270640    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:01.270716 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270669    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-run-ovn\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.270716 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270696    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4fq7\" (UniqueName: \"kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7\") pod \"network-check-target-qcl5s\" (UID: \"8a92e62e-b631-4596-8efc-b4e65ff8e290\") " pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:01.270932 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270720    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-cnibin\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.270932 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270744    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-os-release\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.270932 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270769    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-lib-modules\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.270932 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270795    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9clfm\" (UniqueName: \"kubernetes.io/projected/063466de-9773-499e-b91c-16f818441006-kube-api-access-9clfm\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.270932 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270830    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-cni-netd\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.270932 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4028a654-3327-4719-b4d0-6434b1573bbd-kubelet-config\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:01.270932 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270883    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-run-ovn-kubernetes\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.270932 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270908    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-registration-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.270932 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270932    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-cni-dir\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270958    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-run-k8s-cni-cncf-io\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.270984    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-run-netns\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271009    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-var-lib-cni-bin\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271037    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271081    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zj6s\" (UniqueName: \"kubernetes.io/projected/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-kube-api-access-7zj6s\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271110    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aee74d7d-4f10-43b5-b805-90cd335b79fd-hosts-file\") pod \"node-resolver-2m75c\" (UID: \"aee74d7d-4f10-43b5-b805-90cd335b79fd\") " pod="openshift-dns/node-resolver-2m75c"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271144    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-cni-binary-copy\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271171    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wcr4\" (UniqueName: \"kubernetes.io/projected/9ad78daf-e4fe-4ef6-b833-308bd3211ea3-kube-api-access-9wcr4\") pod \"node-ca-swxbg\" (UID: \"9ad78daf-e4fe-4ef6-b833-308bd3211ea3\") " pod="openshift-image-registry/node-ca-swxbg"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-conf-dir\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271253    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271287    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-modprobe-d\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271318    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-env-overrides\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271343    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aee74d7d-4f10-43b5-b805-90cd335b79fd-tmp-dir\") pod \"node-resolver-2m75c\" (UID: \"aee74d7d-4f10-43b5-b805-90cd335b79fd\") " pod="openshift-dns/node-resolver-2m75c"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271387    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271412    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4028a654-3327-4719-b4d0-6434b1573bbd-dbus\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:01.272782 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271594    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aee74d7d-4f10-43b5-b805-90cd335b79fd-hosts-file\") pod \"node-resolver-2m75c\" (UID: \"aee74d7d-4f10-43b5-b805-90cd335b79fd\") " pod="openshift-dns/node-resolver-2m75c"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.271999    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-run-openvswitch\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272034    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-ovn-node-metrics-cert\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272063    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-socket-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272088    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b75f10f-a52b-469e-a350-6e750c24fbc0-agent-certs\") pod \"konnectivity-agent-gfhzh\" (UID: \"3b75f10f-a52b-469e-a350-6e750c24fbc0\") " pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-run-multus-certs\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272153    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-hostroot\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272170    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aee74d7d-4f10-43b5-b805-90cd335b79fd-tmp-dir\") pod \"node-resolver-2m75c\" (UID: \"aee74d7d-4f10-43b5-b805-90cd335b79fd\") " pod="openshift-dns/node-resolver-2m75c"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272178    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-cnibin\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272207    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-kubelet\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272283    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-ovnkube-config\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272313    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272370    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272399    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4hzq\" (UniqueName: \"kubernetes.io/projected/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-kube-api-access-v4hzq\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272433    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-systemd-units\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272456    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-run-netns\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.274178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272481    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-var-lib-kubelet\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272526    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-ovnkube-script-lib\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272555    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzg9d\" (UniqueName: \"kubernetes.io/projected/e6d626fe-bf36-4e06-a3ff-15e614b8458b-kube-api-access-kzg9d\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272591    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ad78daf-e4fe-4ef6-b833-308bd3211ea3-host\") pod \"node-ca-swxbg\" (UID: \"9ad78daf-e4fe-4ef6-b833-308bd3211ea3\") " pod="openshift-image-registry/node-ca-swxbg"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272632    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-var-lib-kubelet\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272639    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ad78daf-e4fe-4ef6-b833-308bd3211ea3-host\") pod \"node-ca-swxbg\" (UID: \"9ad78daf-e4fe-4ef6-b833-308bd3211ea3\") " pod="openshift-image-registry/node-ca-swxbg"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272704    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-daemon-config\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272742    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-host\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272773    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-slash\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272790    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-run-systemd\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272810    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-device-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272826    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-socket-dir-parent\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qppl7\" (UniqueName: \"kubernetes.io/projected/e42b90e2-e049-49a7-abdb-8e9deaa6cc33-kube-api-access-qppl7\") pod \"iptables-alerter-kd2kj\" (UID: \"e42b90e2-e049-49a7-abdb-8e9deaa6cc33\") " pod="openshift-network-operator/iptables-alerter-kd2kj"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272856    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-sysconfig\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272883    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-kubernetes\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272898    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-sysctl-d\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272913    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-etc-openvswitch\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.275059 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272936    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-node-log\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272959    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-cni-bin\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.272984    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e42b90e2-e049-49a7-abdb-8e9deaa6cc33-iptables-alerter-script\") pod \"iptables-alerter-kd2kj\" (UID: \"e42b90e2-e049-49a7-abdb-8e9deaa6cc33\") " pod="openshift-network-operator/iptables-alerter-kd2kj"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273008    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-sysctl-conf\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273033    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/063466de-9773-499e-b91c-16f818441006-tmp\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273074    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-var-lib-openvswitch\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273101    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-system-cni-dir\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273126    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-systemd\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273151    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-sys\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273176    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-log-socket\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273204    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ad78daf-e4fe-4ef6-b833-308bd3211ea3-serviceca\") pod \"node-ca-swxbg\" (UID: \"9ad78daf-e4fe-4ef6-b833-308bd3211ea3\") " pod="openshift-image-registry/node-ca-swxbg"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273249    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-var-lib-cni-multus\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273275    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/063466de-9773-499e-b91c-16f818441006-etc-tuned\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273298    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-etc-selinux\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273322    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-sys-fs\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273371    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-run\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273413    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-etc-kubernetes\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.275967 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273451    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l6j9\" (UniqueName: \"kubernetes.io/projected/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-kube-api-access-2l6j9\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.276830 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273487    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-os-release\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.276830 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273517    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e42b90e2-e049-49a7-abdb-8e9deaa6cc33-host-slash\") pod \"iptables-alerter-kd2kj\" (UID: \"e42b90e2-e049-49a7-abdb-8e9deaa6cc33\") " pod="openshift-network-operator/iptables-alerter-kd2kj"
Jul 28 19:47:01.276830 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273558    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-system-cni-dir\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.276830 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273588    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b75f10f-a52b-469e-a350-6e750c24fbc0-konnectivity-ca\") pod \"konnectivity-agent-gfhzh\" (UID: \"3b75f10f-a52b-469e-a350-6e750c24fbc0\") " pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:01.276830 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273616    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.276830 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.273976    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ad78daf-e4fe-4ef6-b833-308bd3211ea3-serviceca\") pod \"node-ca-swxbg\" (UID: \"9ad78daf-e4fe-4ef6-b833-308bd3211ea3\") " pod="openshift-image-registry/node-ca-swxbg"
Jul 28 19:47:01.280062 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.279281    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 19:47:01.280062 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.279802    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:47:01.280062 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.279823    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:47:01.280062 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.279837    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q4fq7 for pod openshift-network-diagnostics/network-check-target-qcl5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:01.280062 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.279907    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7 podName:8a92e62e-b631-4596-8efc-b4e65ff8e290 nodeName:}" failed. No retries permitted until 2026-07-28 19:47:01.779876069 +0000 UTC m=+2.960393111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q4fq7" (UniqueName: "kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7") pod "network-check-target-qcl5s" (UID: "8a92e62e-b631-4596-8efc-b4e65ff8e290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:01.285649 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.285583    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxm2w\" (UniqueName: \"kubernetes.io/projected/aee74d7d-4f10-43b5-b805-90cd335b79fd-kube-api-access-kxm2w\") pod \"node-resolver-2m75c\" (UID: \"aee74d7d-4f10-43b5-b805-90cd335b79fd\") " pod="openshift-dns/node-resolver-2m75c"
Jul 28 19:47:01.287673 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.287649    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wcr4\" (UniqueName: \"kubernetes.io/projected/9ad78daf-e4fe-4ef6-b833-308bd3211ea3-kube-api-access-9wcr4\") pod \"node-ca-swxbg\" (UID: \"9ad78daf-e4fe-4ef6-b833-308bd3211ea3\") " pod="openshift-image-registry/node-ca-swxbg"
Jul 28 19:47:01.366994 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.366931    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal" event={"ID":"f39b57f429370ddf00b5bf9b9f3d71ea","Type":"ContainerStarted","Data":"72f2494068b5958681c99fb8ae34098e760b65d8bb24d1b10407ac683afcceae"}
Jul 28 19:47:01.368225 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.368195    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal" event={"ID":"1450dc71fe2617f4b2c6cd952b92073e","Type":"ContainerStarted","Data":"9b7deb250b45a4d9c0917c467f88d2981713705f15beb31b5fc949b82ab1604d"}
Jul 28 19:47:01.372234 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.372210    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:47:01.374879 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.374851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-system-cni-dir\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.374983 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.374893    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-systemd\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.374983 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.374921    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-sys\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.374983 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.374946    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-log-socket\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.374983 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.374973    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-var-lib-cni-multus\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.375207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.374997    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/063466de-9773-499e-b91c-16f818441006-etc-tuned\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.375207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375021    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-etc-selinux\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.375207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375046    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-sys-fs\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.375207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375069    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-run\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.375207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375092    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-etc-kubernetes\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.375207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375116    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l6j9\" (UniqueName: \"kubernetes.io/projected/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-kube-api-access-2l6j9\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.375207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375140    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-os-release\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.375207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375170    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e42b90e2-e049-49a7-abdb-8e9deaa6cc33-host-slash\") pod \"iptables-alerter-kd2kj\" (UID: \"e42b90e2-e049-49a7-abdb-8e9deaa6cc33\") " pod="openshift-network-operator/iptables-alerter-kd2kj"
Jul 28 19:47:01.375207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375195    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-system-cni-dir\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375220    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b75f10f-a52b-469e-a350-6e750c24fbc0-konnectivity-ca\") pod \"konnectivity-agent-gfhzh\" (UID: \"3b75f10f-a52b-469e-a350-6e750c24fbc0\") " pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375244    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375271    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s48k7\" (UniqueName: \"kubernetes.io/projected/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-kube-api-access-s48k7\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375301    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375351    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-run-ovn\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375389    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-cnibin\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375402    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-log-socket\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375413    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-os-release\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375419    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-sys-fs\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375443    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-lib-modules\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375469    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9clfm\" (UniqueName: \"kubernetes.io/projected/063466de-9773-499e-b91c-16f818441006-kube-api-access-9clfm\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375477    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-var-lib-cni-multus\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375480    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-system-cni-dir\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375520    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-systemd\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375531    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-cni-netd\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375535    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-etc-kubernetes\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.375674 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375579    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-run\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.375707    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.375777    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret podName:4028a654-3327-4719-b4d0-6434b1573bbd nodeName:}" failed. No retries permitted until 2026-07-28 19:47:01.875757413 +0000 UTC m=+3.056274463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret") pod "global-pull-secret-syncer-hcb5b" (UID: "4028a654-3327-4719-b4d0-6434b1573bbd") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375797    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-run-ovn\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375860    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-cnibin\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375924    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-os-release\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376008    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-os-release\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376048    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-lib-modules\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376056    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e42b90e2-e049-49a7-abdb-8e9deaa6cc33-host-slash\") pod \"iptables-alerter-kd2kj\" (UID: \"e42b90e2-e049-49a7-abdb-8e9deaa6cc33\") " pod="openshift-network-operator/iptables-alerter-kd2kj"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376068    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-etc-selinux\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376100    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-system-cni-dir\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376151    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.375492    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-cni-netd\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376266    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4028a654-3327-4719-b4d0-6434b1573bbd-kubelet-config\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376293    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-run-ovn-kubernetes\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376318    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-registration-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376337    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4028a654-3327-4719-b4d0-6434b1573bbd-kubelet-config\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:01.376472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376343    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-cni-dir\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376454    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-run-k8s-cni-cncf-io\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376487    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-run-netns\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376514    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-var-lib-cni-bin\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376523    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-cni-dir\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376540    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376566    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zj6s\" (UniqueName: \"kubernetes.io/projected/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-kube-api-access-7zj6s\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376578    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-run-netns\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376582    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-sys\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376593    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-cni-binary-copy\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376617    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-run-ovn-kubernetes\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376621    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376622    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-run-k8s-cni-cncf-io\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376676    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-var-lib-cni-bin\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376691    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376697    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3b75f10f-a52b-469e-a350-6e750c24fbc0-konnectivity-ca\") pod \"konnectivity-agent-gfhzh\" (UID: \"3b75f10f-a52b-469e-a350-6e750c24fbc0\") " pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376724    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-registration-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.377411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376718    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-conf-dir\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376766    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.376842    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376834    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-modprobe-d\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.376911    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs podName:6c6e1a80-f190-4d1d-9e27-a00b79bee30c nodeName:}" failed. No retries permitted until 2026-07-28 19:47:01.876887228 +0000 UTC m=+3.057404280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs") pod "network-metrics-daemon-vwvtz" (UID: "6c6e1a80-f190-4d1d-9e27-a00b79bee30c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376932    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-env-overrides\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.376985    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-modprobe-d\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377025    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-conf-dir\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377070    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377310    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-cni-binary-copy\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377380    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4028a654-3327-4719-b4d0-6434b1573bbd-dbus\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377421    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-env-overrides\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377442    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377457    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-run-openvswitch\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377496    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-ovn-node-metrics-cert\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377500    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4028a654-3327-4719-b4d0-6434b1573bbd-dbus\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377501    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-run-openvswitch\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.378276 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377532    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-socket-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377584    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377623    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b75f10f-a52b-469e-a350-6e750c24fbc0-agent-certs\") pod \"konnectivity-agent-gfhzh\" (UID: \"3b75f10f-a52b-469e-a350-6e750c24fbc0\") " pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377666    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-run-multus-certs\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377691    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-hostroot\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377696    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-socket-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377744    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-hostroot\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377716    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-cnibin\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-kubelet\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377791    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-cnibin\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377824    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-kubelet\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377829    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-run-multus-certs\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377855    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-ovnkube-config\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.377907    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378022    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378124    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378349    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-ovnkube-config\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.379183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378437    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4hzq\" (UniqueName: \"kubernetes.io/projected/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-kube-api-access-v4hzq\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378514    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-systemd-units\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378515    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378576    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-systemd-units\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378611    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-run-netns\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378651    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-run-netns\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378679    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-var-lib-kubelet\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378726    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-var-lib-kubelet\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378754    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-ovnkube-script-lib\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzg9d\" (UniqueName: \"kubernetes.io/projected/e6d626fe-bf36-4e06-a3ff-15e614b8458b-kube-api-access-kzg9d\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378954    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-var-lib-kubelet\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.378980    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-daemon-config\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379004    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-host\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-slash\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379052    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-run-systemd\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379079    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-device-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379105    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-socket-dir-parent\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.380043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379133    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qppl7\" (UniqueName: \"kubernetes.io/projected/e42b90e2-e049-49a7-abdb-8e9deaa6cc33-kube-api-access-qppl7\") pod \"iptables-alerter-kd2kj\" (UID: \"e42b90e2-e049-49a7-abdb-8e9deaa6cc33\") " pod="openshift-network-operator/iptables-alerter-kd2kj"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379157    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-sysconfig\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379180    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-kubernetes\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379196    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-ovnkube-script-lib\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379204    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-sysctl-d\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379229    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-etc-openvswitch\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-run-systemd\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-node-log\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379278    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-cni-bin\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379301    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e42b90e2-e049-49a7-abdb-8e9deaa6cc33-iptables-alerter-script\") pod \"iptables-alerter-kd2kj\" (UID: \"e42b90e2-e049-49a7-abdb-8e9deaa6cc33\") " pod="openshift-network-operator/iptables-alerter-kd2kj"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-sysctl-conf\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379364    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/063466de-9773-499e-b91c-16f818441006-tmp\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379388    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-var-lib-openvswitch\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379435    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-host-var-lib-kubelet\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379471    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-var-lib-openvswitch\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379531    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6d626fe-bf36-4e06-a3ff-15e614b8458b-device-dir\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379588    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-socket-dir-parent\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379794    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-host\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.380972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379852    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-node-log\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.382010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379866    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-cni-bin\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.382010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379892    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-multus-daemon-config\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.382010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379908    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-sysconfig\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.382010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379942    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-host-slash\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.382010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.379958    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-kubernetes\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.382010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.380065    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-sysctl-d\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.382010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.380070    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/063466de-9773-499e-b91c-16f818441006-etc-sysctl-conf\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.382010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.380111    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-etc-openvswitch\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.382010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.380395    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e42b90e2-e049-49a7-abdb-8e9deaa6cc33-iptables-alerter-script\") pod \"iptables-alerter-kd2kj\" (UID: \"e42b90e2-e049-49a7-abdb-8e9deaa6cc33\") " pod="openshift-network-operator/iptables-alerter-kd2kj"
Jul 28 19:47:01.384468 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.384440    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/063466de-9773-499e-b91c-16f818441006-tmp\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.386645 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.386620    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9clfm\" (UniqueName: \"kubernetes.io/projected/063466de-9773-499e-b91c-16f818441006-kube-api-access-9clfm\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.387122 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.387090    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4hzq\" (UniqueName: \"kubernetes.io/projected/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-kube-api-access-v4hzq\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:01.387711 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.387660    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s48k7\" (UniqueName: \"kubernetes.io/projected/8893a3f0-c25c-4f4b-906b-be6e0cf425c9-kube-api-access-s48k7\") pod \"multus-additional-cni-plugins-2hqwp\" (UID: \"8893a3f0-c25c-4f4b-906b-be6e0cf425c9\") " pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.388316 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.388262    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/063466de-9773-499e-b91c-16f818441006-etc-tuned\") pod \"tuned-q7wzh\" (UID: \"063466de-9773-499e-b91c-16f818441006\") " pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.389068 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.389018    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zj6s\" (UniqueName: \"kubernetes.io/projected/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-kube-api-access-7zj6s\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.389463 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.389436    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f6fc39a-c1a5-44f7-94a2-099eb992e02d-ovn-node-metrics-cert\") pod \"ovnkube-node-5m4m9\" (UID: \"9f6fc39a-c1a5-44f7-94a2-099eb992e02d\") " pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.389570 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.389547    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qppl7\" (UniqueName: \"kubernetes.io/projected/e42b90e2-e049-49a7-abdb-8e9deaa6cc33-kube-api-access-qppl7\") pod \"iptables-alerter-kd2kj\" (UID: \"e42b90e2-e049-49a7-abdb-8e9deaa6cc33\") " pod="openshift-network-operator/iptables-alerter-kd2kj"
Jul 28 19:47:01.389998 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.389953    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzg9d\" (UniqueName: \"kubernetes.io/projected/e6d626fe-bf36-4e06-a3ff-15e614b8458b-kube-api-access-kzg9d\") pod \"aws-ebs-csi-driver-node-cmb6q\" (UID: \"e6d626fe-bf36-4e06-a3ff-15e614b8458b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.390708 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.390685    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l6j9\" (UniqueName: \"kubernetes.io/projected/aead52ec-e2c4-4856-a1f3-ff6be89bd3f7-kube-api-access-2l6j9\") pod \"multus-qjrvq\" (UID: \"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7\") " pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.392340 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.392317    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3b75f10f-a52b-469e-a350-6e750c24fbc0-agent-certs\") pod \"konnectivity-agent-gfhzh\" (UID: \"3b75f10f-a52b-469e-a350-6e750c24fbc0\") " pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:01.469583 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.469545    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2m75c"
Jul 28 19:47:01.478269 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.478237    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-swxbg"
Jul 28 19:47:01.484623 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:01.484585    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee74d7d_4f10_43b5_b805_90cd335b79fd.slice/crio-492f305ac20075d8ec1d9ad0da030e23a5d002230acd61ddd6c8be8c7f1170e7 WatchSource:0}: Error finding container 492f305ac20075d8ec1d9ad0da030e23a5d002230acd61ddd6c8be8c7f1170e7: Status 404 returned error can't find the container with id 492f305ac20075d8ec1d9ad0da030e23a5d002230acd61ddd6c8be8c7f1170e7
Jul 28 19:47:01.489716 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:01.489682    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ad78daf_e4fe_4ef6_b833_308bd3211ea3.slice/crio-3ee20aee389347637ee9e23032b8d7f9cdfc98be528a4a674480dbc133870d79 WatchSource:0}: Error finding container 3ee20aee389347637ee9e23032b8d7f9cdfc98be528a4a674480dbc133870d79: Status 404 returned error can't find the container with id 3ee20aee389347637ee9e23032b8d7f9cdfc98be528a4a674480dbc133870d79
Jul 28 19:47:01.491171 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.491150    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kd2kj"
Jul 28 19:47:01.496975 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.496952    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2hqwp"
Jul 28 19:47:01.504500 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:01.504472    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42b90e2_e049_49a7_abdb_8e9deaa6cc33.slice/crio-c8ab112092c295c04ddafa13d38b64fd52a60262790e09ce7b3add3543a89a64 WatchSource:0}: Error finding container c8ab112092c295c04ddafa13d38b64fd52a60262790e09ce7b3add3543a89a64: Status 404 returned error can't find the container with id c8ab112092c295c04ddafa13d38b64fd52a60262790e09ce7b3add3543a89a64
Jul 28 19:47:01.505700 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.505593    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:01.509350 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:01.509333    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8893a3f0_c25c_4f4b_906b_be6e0cf425c9.slice/crio-15943c1746e63b5d6c21fc29cdd977fb40ecc44e596305f93c4ee6ed355f448b WatchSource:0}: Error finding container 15943c1746e63b5d6c21fc29cdd977fb40ecc44e596305f93c4ee6ed355f448b: Status 404 returned error can't find the container with id 15943c1746e63b5d6c21fc29cdd977fb40ecc44e596305f93c4ee6ed355f448b
Jul 28 19:47:01.513513 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.513489    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q"
Jul 28 19:47:01.517787 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:01.517760    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6fc39a_c1a5_44f7_94a2_099eb992e02d.slice/crio-726e7581aab8b86a758de706cb523db90f5fa34c3e0e8440d91ecd08b412db12 WatchSource:0}: Error finding container 726e7581aab8b86a758de706cb523db90f5fa34c3e0e8440d91ecd08b412db12: Status 404 returned error can't find the container with id 726e7581aab8b86a758de706cb523db90f5fa34c3e0e8440d91ecd08b412db12
Jul 28 19:47:01.521056 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.520875    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q7wzh"
Jul 28 19:47:01.524072 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:01.524031    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d626fe_bf36_4e06_a3ff_15e614b8458b.slice/crio-08bc86397f6398de001b58f1922227622e3eee9cb406bf3975f08fd91b764f6f WatchSource:0}: Error finding container 08bc86397f6398de001b58f1922227622e3eee9cb406bf3975f08fd91b764f6f: Status 404 returned error can't find the container with id 08bc86397f6398de001b58f1922227622e3eee9cb406bf3975f08fd91b764f6f
Jul 28 19:47:01.531751 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.531726    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:01.533195 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:01.533170    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063466de_9773_499e_b91c_16f818441006.slice/crio-21e4cf2ba2569779fd94ae0c66b1628bca2308f0224f9ac7029b6b6149e91bdb WatchSource:0}: Error finding container 21e4cf2ba2569779fd94ae0c66b1628bca2308f0224f9ac7029b6b6149e91bdb: Status 404 returned error can't find the container with id 21e4cf2ba2569779fd94ae0c66b1628bca2308f0224f9ac7029b6b6149e91bdb
Jul 28 19:47:01.540450 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.540426    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qjrvq"
Jul 28 19:47:01.543959 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:01.543935    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b75f10f_a52b_469e_a350_6e750c24fbc0.slice/crio-23ecd34a4cb42210b777340185bbcd6aa8631acef044dad54cfa7a097d089b37 WatchSource:0}: Error finding container 23ecd34a4cb42210b777340185bbcd6aa8631acef044dad54cfa7a097d089b37: Status 404 returned error can't find the container with id 23ecd34a4cb42210b777340185bbcd6aa8631acef044dad54cfa7a097d089b37
Jul 28 19:47:01.550848 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:01.550823    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaead52ec_e2c4_4856_a1f3_ff6be89bd3f7.slice/crio-ed51d05e2ff0525dec9e9797e554e10c4500f91abd9adf7a0863e13c01cd2374 WatchSource:0}: Error finding container ed51d05e2ff0525dec9e9797e554e10c4500f91abd9adf7a0863e13c01cd2374: Status 404 returned error can't find the container with id ed51d05e2ff0525dec9e9797e554e10c4500f91abd9adf7a0863e13c01cd2374
Jul 28 19:47:01.781960 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.781855    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4fq7\" (UniqueName: \"kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7\") pod \"network-check-target-qcl5s\" (UID: \"8a92e62e-b631-4596-8efc-b4e65ff8e290\") " pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:01.782198 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.782030    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:47:01.782198 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.782061    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:47:01.782198 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.782076    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q4fq7 for pod openshift-network-diagnostics/network-check-target-qcl5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:01.782198 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.782154    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7 podName:8a92e62e-b631-4596-8efc-b4e65ff8e290 nodeName:}" failed. No retries permitted until 2026-07-28 19:47:02.782132621 +0000 UTC m=+3.962649666 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q4fq7" (UniqueName: "kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7") pod "network-check-target-qcl5s" (UID: "8a92e62e-b631-4596-8efc-b4e65ff8e290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:01.883081 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.883041    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:01.883248 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:01.883100    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:01.883248 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.883227    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:01.883248 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.883244    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:01.883452 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.883298    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs podName:6c6e1a80-f190-4d1d-9e27-a00b79bee30c nodeName:}" failed. No retries permitted until 2026-07-28 19:47:02.88328468 +0000 UTC m=+4.063801720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs") pod "network-metrics-daemon-vwvtz" (UID: "6c6e1a80-f190-4d1d-9e27-a00b79bee30c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:01.883452 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:01.883319    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret podName:4028a654-3327-4719-b4d0-6434b1573bbd nodeName:}" failed. No retries permitted until 2026-07-28 19:47:02.883306985 +0000 UTC m=+4.063824025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret") pod "global-pull-secret-syncer-hcb5b" (UID: "4028a654-3327-4719-b4d0-6434b1573bbd") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:02.207364 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.207321    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:42:01 +0000 UTC" deadline="2028-02-12 08:49:31.880841059 +0000 UTC"
Jul 28 19:47:02.207364 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.207362    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13525h2m29.673485078s"
Jul 28 19:47:02.360508 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.360473    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:02.360681 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.360473    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:02.360681 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:02.360615    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:02.360801 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:02.360681    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:02.371858 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.371817    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qjrvq" event={"ID":"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7","Type":"ContainerStarted","Data":"ed51d05e2ff0525dec9e9797e554e10c4500f91abd9adf7a0863e13c01cd2374"}
Jul 28 19:47:02.373454 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.373401    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gfhzh" event={"ID":"3b75f10f-a52b-469e-a350-6e750c24fbc0","Type":"ContainerStarted","Data":"23ecd34a4cb42210b777340185bbcd6aa8631acef044dad54cfa7a097d089b37"}
Jul 28 19:47:02.374731 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.374706    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hqwp" event={"ID":"8893a3f0-c25c-4f4b-906b-be6e0cf425c9","Type":"ContainerStarted","Data":"15943c1746e63b5d6c21fc29cdd977fb40ecc44e596305f93c4ee6ed355f448b"}
Jul 28 19:47:02.376101 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.376073    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-swxbg" event={"ID":"9ad78daf-e4fe-4ef6-b833-308bd3211ea3","Type":"ContainerStarted","Data":"3ee20aee389347637ee9e23032b8d7f9cdfc98be528a4a674480dbc133870d79"}
Jul 28 19:47:02.377394 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.377368    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q7wzh" event={"ID":"063466de-9773-499e-b91c-16f818441006","Type":"ContainerStarted","Data":"21e4cf2ba2569779fd94ae0c66b1628bca2308f0224f9ac7029b6b6149e91bdb"}
Jul 28 19:47:02.378789 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.378765    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q" event={"ID":"e6d626fe-bf36-4e06-a3ff-15e614b8458b","Type":"ContainerStarted","Data":"08bc86397f6398de001b58f1922227622e3eee9cb406bf3975f08fd91b764f6f"}
Jul 28 19:47:02.380146 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.380124    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9" event={"ID":"9f6fc39a-c1a5-44f7-94a2-099eb992e02d","Type":"ContainerStarted","Data":"726e7581aab8b86a758de706cb523db90f5fa34c3e0e8440d91ecd08b412db12"}
Jul 28 19:47:02.381493 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.381459    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kd2kj" event={"ID":"e42b90e2-e049-49a7-abdb-8e9deaa6cc33","Type":"ContainerStarted","Data":"c8ab112092c295c04ddafa13d38b64fd52a60262790e09ce7b3add3543a89a64"}
Jul 28 19:47:02.382826 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.382802    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2m75c" event={"ID":"aee74d7d-4f10-43b5-b805-90cd335b79fd","Type":"ContainerStarted","Data":"492f305ac20075d8ec1d9ad0da030e23a5d002230acd61ddd6c8be8c7f1170e7"}
Jul 28 19:47:02.791722 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.791671    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4fq7\" (UniqueName: \"kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7\") pod \"network-check-target-qcl5s\" (UID: \"8a92e62e-b631-4596-8efc-b4e65ff8e290\") " pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:02.791900 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:02.791854    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:47:02.791900 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:02.791878    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:47:02.791900 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:02.791900    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q4fq7 for pod openshift-network-diagnostics/network-check-target-qcl5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:02.792082 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:02.791965    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7 podName:8a92e62e-b631-4596-8efc-b4e65ff8e290 nodeName:}" failed. No retries permitted until 2026-07-28 19:47:04.791947916 +0000 UTC m=+5.972464956 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q4fq7" (UniqueName: "kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7") pod "network-check-target-qcl5s" (UID: "8a92e62e-b631-4596-8efc-b4e65ff8e290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:02.892553 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.892511    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:02.892731 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:02.892614    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:02.892792 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:02.892765    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:02.892918 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:02.892836    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret podName:4028a654-3327-4719-b4d0-6434b1573bbd nodeName:}" failed. No retries permitted until 2026-07-28 19:47:04.892814997 +0000 UTC m=+6.073332060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret") pod "global-pull-secret-syncer-hcb5b" (UID: "4028a654-3327-4719-b4d0-6434b1573bbd") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:02.893280 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:02.893261    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:02.893407 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:02.893393    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs podName:6c6e1a80-f190-4d1d-9e27-a00b79bee30c nodeName:}" failed. No retries permitted until 2026-07-28 19:47:04.89333439 +0000 UTC m=+6.073851448 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs") pod "network-metrics-daemon-vwvtz" (UID: "6c6e1a80-f190-4d1d-9e27-a00b79bee30c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:03.208002 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:03.207955    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:42:01 +0000 UTC" deadline="2028-02-25 01:20:58.77816651 +0000 UTC"
Jul 28 19:47:03.208002 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:03.207997    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13829h33m55.570175881s"
Jul 28 19:47:03.370539 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:03.370481    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:03.370709 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:03.370655    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:03.393529 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:03.393313    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal" event={"ID":"f39b57f429370ddf00b5bf9b9f3d71ea","Type":"ContainerStarted","Data":"aa17390090d8976e46a1a495856dc753f554de389c26fca33184852a73f9f12e"}
Jul 28 19:47:04.360994 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:04.360287    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:04.360994 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:04.360454    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:04.360994 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:04.360853    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:04.360994 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:04.360947    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:04.411475 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:04.411441    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal" event={"ID":"1450dc71fe2617f4b2c6cd952b92073e","Type":"ContainerStarted","Data":"13e5c037ae923c3f7c39559d760c124dc2953cbe715cdfe28d79b2713547186c"}
Jul 28 19:47:04.426152 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:04.426092    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-138.ec2.internal" podStartSLOduration=3.426071517 podStartE2EDuration="3.426071517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:47:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:03.406889006 +0000 UTC m=+4.587406065" watchObservedRunningTime="2026-07-28 19:47:04.426071517 +0000 UTC m=+5.606588580"
Jul 28 19:47:04.808405 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:04.807741    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4fq7\" (UniqueName: \"kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7\") pod \"network-check-target-qcl5s\" (UID: \"8a92e62e-b631-4596-8efc-b4e65ff8e290\") " pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:04.808405 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:04.807948    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:47:04.808405 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:04.807969    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:47:04.808405 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:04.807982    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q4fq7 for pod openshift-network-diagnostics/network-check-target-qcl5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:04.808405 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:04.808043    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7 podName:8a92e62e-b631-4596-8efc-b4e65ff8e290 nodeName:}" failed. No retries permitted until 2026-07-28 19:47:08.808023704 +0000 UTC m=+9.988540746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q4fq7" (UniqueName: "kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7") pod "network-check-target-qcl5s" (UID: "8a92e62e-b631-4596-8efc-b4e65ff8e290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:04.909045 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:04.908992    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:04.909229 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:04.909059    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:04.909229 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:04.909148    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:04.909229 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:04.909170    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:04.909229 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:04.909211    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret podName:4028a654-3327-4719-b4d0-6434b1573bbd nodeName:}" failed. No retries permitted until 2026-07-28 19:47:08.909191445 +0000 UTC m=+10.089708494 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret") pod "global-pull-secret-syncer-hcb5b" (UID: "4028a654-3327-4719-b4d0-6434b1573bbd") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:04.909229 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:04.909232    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs podName:6c6e1a80-f190-4d1d-9e27-a00b79bee30c nodeName:}" failed. No retries permitted until 2026-07-28 19:47:08.909221092 +0000 UTC m=+10.089738133 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs") pod "network-metrics-daemon-vwvtz" (UID: "6c6e1a80-f190-4d1d-9e27-a00b79bee30c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:05.361127 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:05.360606    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:05.361127 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:05.360751    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:06.360206 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:06.360173    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:06.360405 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:06.360336    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:06.360572 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:06.360551    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:06.360702 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:06.360680    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:07.360094 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:07.359954    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:07.360094 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:07.360083    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:07.420145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:07.419402    2569 generic.go:356] "Generic (PLEG): container finished" podID="1450dc71fe2617f4b2c6cd952b92073e" containerID="13e5c037ae923c3f7c39559d760c124dc2953cbe715cdfe28d79b2713547186c" exitCode=0
Jul 28 19:47:07.420145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:07.419458    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal" event={"ID":"1450dc71fe2617f4b2c6cd952b92073e","Type":"ContainerDied","Data":"13e5c037ae923c3f7c39559d760c124dc2953cbe715cdfe28d79b2713547186c"}
Jul 28 19:47:08.360552 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:08.360040    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:08.360552 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:08.360182    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:08.360552 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:08.360390    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:08.360552 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:08.360471    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:08.840762 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:08.840699    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4fq7\" (UniqueName: \"kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7\") pod \"network-check-target-qcl5s\" (UID: \"8a92e62e-b631-4596-8efc-b4e65ff8e290\") " pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:08.840947 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:08.840862    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:47:08.840947 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:08.840888    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:47:08.840947 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:08.840902    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q4fq7 for pod openshift-network-diagnostics/network-check-target-qcl5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:08.841054 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:08.840965    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7 podName:8a92e62e-b631-4596-8efc-b4e65ff8e290 nodeName:}" failed. No retries permitted until 2026-07-28 19:47:16.840946553 +0000 UTC m=+18.021463588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q4fq7" (UniqueName: "kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7") pod "network-check-target-qcl5s" (UID: "8a92e62e-b631-4596-8efc-b4e65ff8e290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:08.941239 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:08.941203    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:08.941420 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:08.941266    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:08.941420 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:08.941402    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:08.941591 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:08.941461    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:08.941591 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:08.941464    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs podName:6c6e1a80-f190-4d1d-9e27-a00b79bee30c nodeName:}" failed. No retries permitted until 2026-07-28 19:47:16.941445425 +0000 UTC m=+18.121962459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs") pod "network-metrics-daemon-vwvtz" (UID: "6c6e1a80-f190-4d1d-9e27-a00b79bee30c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:08.941591 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:08.941548    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret podName:4028a654-3327-4719-b4d0-6434b1573bbd nodeName:}" failed. No retries permitted until 2026-07-28 19:47:16.941518694 +0000 UTC m=+18.122035728 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret") pod "global-pull-secret-syncer-hcb5b" (UID: "4028a654-3327-4719-b4d0-6434b1573bbd") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:09.361390 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:09.360915    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:09.361390 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:09.361024    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:10.360695 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:10.360482    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:10.360695 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:10.360521    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:10.360695 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:10.360625    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:10.360973 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:10.360716    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:11.362474 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:11.361790    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:11.362474 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:11.361904    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:12.360441 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:12.360416    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:12.360441 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:12.360433    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:12.360610 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:12.360512    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:12.360687 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:12.360662    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:13.360238 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:13.360208    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:13.360649 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:13.360354    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:14.359607 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:14.359573    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:14.359771 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:14.359693    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:14.359771 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:14.359737    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:14.359873 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:14.359843    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:15.360148 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:15.360119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:15.360669 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:15.360246    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:16.360296 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:16.360251    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:16.360757 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:16.360305    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:16.360757 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:16.360405    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:16.360757 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:16.360527    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:16.908593 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:16.908555    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4fq7\" (UniqueName: \"kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7\") pod \"network-check-target-qcl5s\" (UID: \"8a92e62e-b631-4596-8efc-b4e65ff8e290\") " pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:16.908753 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:16.908738    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:47:16.908817 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:16.908760    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:47:16.908817 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:16.908771    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q4fq7 for pod openshift-network-diagnostics/network-check-target-qcl5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:16.908959 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:16.908822    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7 podName:8a92e62e-b631-4596-8efc-b4e65ff8e290 nodeName:}" failed. No retries permitted until 2026-07-28 19:47:32.908803486 +0000 UTC m=+34.089320520 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q4fq7" (UniqueName: "kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7") pod "network-check-target-qcl5s" (UID: "8a92e62e-b631-4596-8efc-b4e65ff8e290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:17.009293 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:17.009239    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:17.009441 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:17.009324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:17.009441 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:17.009386    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:17.009441 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:17.009434    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:17.009592 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:17.009457    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret podName:4028a654-3327-4719-b4d0-6434b1573bbd nodeName:}" failed. No retries permitted until 2026-07-28 19:47:33.009436912 +0000 UTC m=+34.189953966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret") pod "global-pull-secret-syncer-hcb5b" (UID: "4028a654-3327-4719-b4d0-6434b1573bbd") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:17.009592 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:17.009477    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs podName:6c6e1a80-f190-4d1d-9e27-a00b79bee30c nodeName:}" failed. No retries permitted until 2026-07-28 19:47:33.00946491 +0000 UTC m=+34.189981944 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs") pod "network-metrics-daemon-vwvtz" (UID: "6c6e1a80-f190-4d1d-9e27-a00b79bee30c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:17.360328 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:17.360273    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:17.360741 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:17.360424    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:18.360377 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:18.360340    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:18.360851 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:18.360352    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:18.360851 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:18.360467    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:18.360851 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:18.360515    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:19.361328 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:19.361295    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:19.361785 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:19.361381    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:20.361317 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.360798    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:20.361317 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:20.361100    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:20.361317 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.361170    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:20.361317 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:20.361256    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:20.444056 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.443972    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gfhzh" event={"ID":"3b75f10f-a52b-469e-a350-6e750c24fbc0","Type":"ContainerStarted","Data":"41b4aa7998db7a7186639f0438f0df0aabc761d06f933968c097eacce6b1f0cf"}
Jul 28 19:47:20.445775 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.445436    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hqwp" event={"ID":"8893a3f0-c25c-4f4b-906b-be6e0cf425c9","Type":"ContainerStarted","Data":"456449cd0301d5780059c2353da24769bf3dc01d3fbbcacd9ad256ba272ed1fc"}
Jul 28 19:47:20.447532 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.447507    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-swxbg" event={"ID":"9ad78daf-e4fe-4ef6-b833-308bd3211ea3","Type":"ContainerStarted","Data":"049914b2df677b1aef976c190ef28481c373072477b824d99f68a99d7756f8fa"}
Jul 28 19:47:20.448961 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.448934    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q7wzh" event={"ID":"063466de-9773-499e-b91c-16f818441006","Type":"ContainerStarted","Data":"04faa397e1e261cd3e90d6aa7c58be2220768d387199b73db94a4cde4701479f"}
Jul 28 19:47:20.450690 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.450665    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q" event={"ID":"e6d626fe-bf36-4e06-a3ff-15e614b8458b","Type":"ContainerStarted","Data":"458bf3d0efb6aa8c5c4627ae493788e330340e531e223c997eeece0dbf664b2b"}
Jul 28 19:47:20.452025 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.451979    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2m75c" event={"ID":"aee74d7d-4f10-43b5-b805-90cd335b79fd","Type":"ContainerStarted","Data":"7d1c9be504396ccfd3ef03c0b7850cdd78e14cc4fb44aed924cdc7a28c41cd3d"}
Jul 28 19:47:20.453598 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.453577    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal" event={"ID":"1450dc71fe2617f4b2c6cd952b92073e","Type":"ContainerStarted","Data":"d7fcb0f93233f9ac0e804ba230bfb4d6d79155c757959044314e5e529b056ac5"}
Jul 28 19:47:20.455137 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.455118    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qjrvq" event={"ID":"aead52ec-e2c4-4856-a1f3-ff6be89bd3f7","Type":"ContainerStarted","Data":"be908b261f282dc8501c4537957017a834daba7ab7935312510bd8473fd4c63b"}
Jul 28 19:47:20.458102 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.458058    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gfhzh" podStartSLOduration=2.897278439 podStartE2EDuration="21.458025259s" totalImagesPullingTime="18.56074682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:20.457999373 +0000 UTC m=+21.638516430" watchObservedRunningTime="2026-07-28 19:47:20.458025259 +0000 UTC m=+21.638542316"
Jul 28 19:47:20.473636 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.473594    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-138.ec2.internal" podStartSLOduration=19.473579926 podStartE2EDuration="19.473579926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:47:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:20.473109934 +0000 UTC m=+21.653626993" watchObservedRunningTime="2026-07-28 19:47:20.473579926 +0000 UTC m=+21.654096982"
Jul 28 19:47:20.488140 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.488103    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qjrvq" podStartSLOduration=2.797037409 podStartE2EDuration="21.488091165s" totalImagesPullingTime="18.691053756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:20.487742598 +0000 UTC m=+21.668259654" watchObservedRunningTime="2026-07-28 19:47:20.488091165 +0000 UTC m=+21.668608238"
Jul 28 19:47:20.500791 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.500755    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2m75c" podStartSLOduration=2.7865567799999997 podStartE2EDuration="21.500739815s" totalImagesPullingTime="18.714183035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:20.500332635 +0000 UTC m=+21.680849694" watchObservedRunningTime="2026-07-28 19:47:20.500739815 +0000 UTC m=+21.681256875"
Jul 28 19:47:20.511596 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.511544    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-swxbg" podStartSLOduration=2.8061211459999997 podStartE2EDuration="21.511532216s" totalImagesPullingTime="18.70541107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:20.511523719 +0000 UTC m=+21.692040771" watchObservedRunningTime="2026-07-28 19:47:20.511532216 +0000 UTC m=+21.692049273"
Jul 28 19:47:20.555134 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.555072    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q7wzh" podStartSLOduration=2.880415133 podStartE2EDuration="21.55505412s" totalImagesPullingTime="18.674638987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:20.555032024 +0000 UTC m=+21.735549080" watchObservedRunningTime="2026-07-28 19:47:20.55505412 +0000 UTC m=+21.735571177"
Jul 28 19:47:20.611347 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.611317    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:20.679707 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:20.679685    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:21.360005 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.359986    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:21.360123 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:21.360105    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:21.458223 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.458193    2569 generic.go:356] "Generic (PLEG): container finished" podID="8893a3f0-c25c-4f4b-906b-be6e0cf425c9" containerID="456449cd0301d5780059c2353da24769bf3dc01d3fbbcacd9ad256ba272ed1fc" exitCode=0
Jul 28 19:47:21.458895 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.458267    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hqwp" event={"ID":"8893a3f0-c25c-4f4b-906b-be6e0cf425c9","Type":"ContainerDied","Data":"456449cd0301d5780059c2353da24769bf3dc01d3fbbcacd9ad256ba272ed1fc"}
Jul 28 19:47:21.460903 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.460886    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 19:47:21.461168 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.461152    2569 generic.go:356] "Generic (PLEG): container finished" podID="9f6fc39a-c1a5-44f7-94a2-099eb992e02d" containerID="248e2f7b49f84eb3a78152486f2c51ae3299ab58f755ef5fc531886b121719ea" exitCode=1
Jul 28 19:47:21.461263 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.461242    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9" event={"ID":"9f6fc39a-c1a5-44f7-94a2-099eb992e02d","Type":"ContainerStarted","Data":"90085580878e9712b60c2ae94ec871389f17ecdf822c9575274798e8db372007"}
Jul 28 19:47:21.461340 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.461326    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9" event={"ID":"9f6fc39a-c1a5-44f7-94a2-099eb992e02d","Type":"ContainerStarted","Data":"a69588a78f723d1d6f5646f98be925551acbacf84e4e705a08e2cdb19798a6ae"}
Jul 28 19:47:21.461383 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.461363    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9" event={"ID":"9f6fc39a-c1a5-44f7-94a2-099eb992e02d","Type":"ContainerStarted","Data":"4503262f023ba23196c8064344cb2d544d3c062f3e29ca8150c9a250bb7023a1"}
Jul 28 19:47:21.461425 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.461397    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9" event={"ID":"9f6fc39a-c1a5-44f7-94a2-099eb992e02d","Type":"ContainerStarted","Data":"3114eff4199201462c554a634d56ba7ef4987d5508da15f9f93d7a4e999667cd"}
Jul 28 19:47:21.461595 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.461565    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9" event={"ID":"9f6fc39a-c1a5-44f7-94a2-099eb992e02d","Type":"ContainerDied","Data":"248e2f7b49f84eb3a78152486f2c51ae3299ab58f755ef5fc531886b121719ea"}
Jul 28 19:47:21.461658 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.461639    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9" event={"ID":"9f6fc39a-c1a5-44f7-94a2-099eb992e02d","Type":"ContainerStarted","Data":"12005a845ace0e33b33f4fd1be3cd50991d998f8912e0de191c95a03b2a665a4"}
Jul 28 19:47:21.658959 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:21.658939    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 19:47:22.228467 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:22.228360    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T19:47:21.658955635Z","UUID":"14a246c6-bfcd-46a8-8852-646b06999acc","Handler":null,"Name":"","Endpoint":""}
Jul 28 19:47:22.231015 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:22.230993    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 19:47:22.231144 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:22.231036    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 19:47:22.360207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:22.360174    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:22.360383 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:22.360218    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:22.360383 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:22.360332    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:22.360515 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:22.360466    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:22.466005 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:22.465967    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q" event={"ID":"e6d626fe-bf36-4e06-a3ff-15e614b8458b","Type":"ContainerStarted","Data":"fe4ed7b5e5d74e1128a8d803fc0fec2665f33fcda2fd9d831f4267107a549254"}
Jul 28 19:47:22.467710 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:22.467672    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kd2kj" event={"ID":"e42b90e2-e049-49a7-abdb-8e9deaa6cc33","Type":"ContainerStarted","Data":"d7bbcf4af251c25d339cd4e0b444110a7af2f617f46a57151376d292e151bcfb"}
Jul 28 19:47:22.481379 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:22.481292    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kd2kj" podStartSLOduration=4.802431054 podStartE2EDuration="23.481260546s" totalImagesPullingTime="18.678829492s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:22.480972884 +0000 UTC m=+23.661489943" watchObservedRunningTime="2026-07-28 19:47:22.481260546 +0000 UTC m=+23.661777607"
Jul 28 19:47:22.597528 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:22.597496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:22.649060 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:22.649032    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:23.360618 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:23.360376    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:23.360867 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:23.360744    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:23.472894 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:23.472861    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q" event={"ID":"e6d626fe-bf36-4e06-a3ff-15e614b8458b","Type":"ContainerStarted","Data":"b017a9bf2743631304a1f15ff7b05e258d8e7fb0c7d6a8ed6c8f3a752be59f79"}
Jul 28 19:47:23.498028 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:23.497990    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cmb6q" podStartSLOduration=3.616765185 podStartE2EDuration="24.497974944s" totalImagesPullingTime="20.881209759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:23.489107168 +0000 UTC m=+24.669624249" watchObservedRunningTime="2026-07-28 19:47:23.497974944 +0000 UTC m=+24.678492003"
Jul 28 19:47:23.556250 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:23.556051    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:23.556677 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:23.556660    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:24.360611 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:24.360576    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:24.360611 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:24.360598    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:24.360852 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:24.360717    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:24.360852 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:24.360804    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:24.478084 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:24.478053    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 19:47:24.478574 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:24.478506    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9" event={"ID":"9f6fc39a-c1a5-44f7-94a2-099eb992e02d","Type":"ContainerStarted","Data":"7e53263505aaa815a6dc540f4d967e84204ac097c6cd9bd2aa2fbf125ba067ca"}
Jul 28 19:47:24.478688 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:24.478660    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:24.479122 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:24.479105    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gfhzh"
Jul 28 19:47:24.535662 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:24.535638    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:25.360400 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:25.360368    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:25.360574 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:25.360489    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:25.599067 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:25.599037    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:25.646363 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:25.646327    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:26.359970 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:26.359788    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:26.360138 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:26.359825    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:26.360138 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:26.360038    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:26.360138 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:26.360114    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:26.484904 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:26.484874    2569 generic.go:356] "Generic (PLEG): container finished" podID="8893a3f0-c25c-4f4b-906b-be6e0cf425c9" containerID="4e8c064763b7cea6e5802f1f3f24e74f1b022c8f774eccc4e38ae7c2c192e7e4" exitCode=0
Jul 28 19:47:26.485042 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:26.484952    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hqwp" event={"ID":"8893a3f0-c25c-4f4b-906b-be6e0cf425c9","Type":"ContainerDied","Data":"4e8c064763b7cea6e5802f1f3f24e74f1b022c8f774eccc4e38ae7c2c192e7e4"}
Jul 28 19:47:26.488414 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:26.488396    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 19:47:26.488784 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:26.488749    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9" event={"ID":"9f6fc39a-c1a5-44f7-94a2-099eb992e02d","Type":"ContainerStarted","Data":"9c8566f6f978cb8a1b79896309d0913faa03d5369bec96e56d52f49b5cfccf6b"}
Jul 28 19:47:26.489181 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:26.489164    2569 scope.go:117] "RemoveContainer" containerID="248e2f7b49f84eb3a78152486f2c51ae3299ab58f755ef5fc531886b121719ea"
Jul 28 19:47:27.031969 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.031937    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:27.111742 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.111712    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:27.186619 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.186588    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:27.287021 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.286952    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:27.360114 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.360084    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:27.360244 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:27.360190    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:27.495788 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.495758    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:27.500679 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.500657    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 19:47:27.501767 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.501734    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9" event={"ID":"9f6fc39a-c1a5-44f7-94a2-099eb992e02d","Type":"ContainerStarted","Data":"c8b44da7e9407ce52a32b351cc81a60987cb5112c6fbc111690dbadd9bc9d14f"}
Jul 28 19:47:27.503629 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.502076    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 19:47:27.503629 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.503447    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:27.503629 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.503546    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:27.522081 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.522056    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:27.522372 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.522347    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:27.534895 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.534821    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9" podStartSLOduration=9.629563939 podStartE2EDuration="28.53480295s" totalImagesPullingTime="18.905239011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:27.534168121 +0000 UTC m=+28.714685194" watchObservedRunningTime="2026-07-28 19:47:27.53480295 +0000 UTC m=+28.715320005"
Jul 28 19:47:27.561103 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.561083    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:27.616513 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.616474    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:27.736983 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.736957    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:27.779615 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.779585    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qcl5s"]
Jul 28 19:47:27.779724 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.779685    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:27.779796 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:27.779776    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:27.782451 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.782428    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vwvtz"]
Jul 28 19:47:27.782569 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.782534    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:27.782628 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:27.782611    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:27.794020 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.793973    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hcb5b"]
Jul 28 19:47:27.794095 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.794051    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:27.794157 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:27.794137    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:27.797591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:27.797558    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:28.246682 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:28.246658    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:28.427510 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:28.427479    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2m75c_aee74d7d-4f10-43b5-b805-90cd335b79fd/dns-node-resolver/0.log"
Jul 28 19:47:28.505468 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:28.505410    2569 generic.go:356] "Generic (PLEG): container finished" podID="8893a3f0-c25c-4f4b-906b-be6e0cf425c9" containerID="d918532b83cff7e88d3dee0f7d2b000a6e29f1e107342d1ac93b6cfeb8cddccc" exitCode=0
Jul 28 19:47:28.505559 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:28.505488    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hqwp" event={"ID":"8893a3f0-c25c-4f4b-906b-be6e0cf425c9","Type":"ContainerDied","Data":"d918532b83cff7e88d3dee0f7d2b000a6e29f1e107342d1ac93b6cfeb8cddccc"}
Jul 28 19:47:28.596508 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:28.596475    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:28.669926 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:28.669877    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:29.340564 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:29.340522    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:29.360327 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:29.360298    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:29.360452 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:29.360397    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:29.360452 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:29.360429    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:29.360564 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:29.360477    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:29.360564 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:29.360495    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:29.360713 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:29.360588    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:29.400541 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:29.400516    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:29.501312 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:29.501271    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:29.626828 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:29.626809    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-swxbg_9ad78daf-e4fe-4ef6-b833-308bd3211ea3/node-ca/0.log"
Jul 28 19:47:29.890631 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:29.890569    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:29.990600 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:29.990574    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:30.162770 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:30.162688    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:30.511881 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:30.511856    2569 generic.go:356] "Generic (PLEG): container finished" podID="8893a3f0-c25c-4f4b-906b-be6e0cf425c9" containerID="d8ca4debeaf99869d6e7c23ed4325a1b9568d90f23d913d6a579d344dbfc62b2" exitCode=0
Jul 28 19:47:30.512443 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:30.511906    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hqwp" event={"ID":"8893a3f0-c25c-4f4b-906b-be6e0cf425c9","Type":"ContainerDied","Data":"d8ca4debeaf99869d6e7c23ed4325a1b9568d90f23d913d6a579d344dbfc62b2"}
Jul 28 19:47:30.788041 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:30.787964    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:30.888289 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:30.888256    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:31.360535 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:31.360503    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:31.360697 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:31.360505    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:31.360697 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:31.360644    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:31.360829 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:31.360726    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:31.360829 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:31.360748    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:31.360942 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:31.360843    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:32.542787 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:32.542754    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:32.925087 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:32.925058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4fq7\" (UniqueName: \"kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7\") pod \"network-check-target-qcl5s\" (UID: \"8a92e62e-b631-4596-8efc-b4e65ff8e290\") " pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:32.925310 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:32.925199    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:47:32.925310 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:32.925214    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:47:32.925310 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:32.925226    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q4fq7 for pod openshift-network-diagnostics/network-check-target-qcl5s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:32.925310 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:32.925295    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7 podName:8a92e62e-b631-4596-8efc-b4e65ff8e290 nodeName:}" failed. No retries permitted until 2026-07-28 19:48:04.925265827 +0000 UTC m=+66.105782877 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-q4fq7" (UniqueName: "kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7") pod "network-check-target-qcl5s" (UID: "8a92e62e-b631-4596-8efc-b4e65ff8e290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:47:33.025475 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:33.025443    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:33.025660 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:33.025501    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:33.025660 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:33.025612    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:33.025660 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:33.025629    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:33.025822 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:33.025672    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs podName:6c6e1a80-f190-4d1d-9e27-a00b79bee30c nodeName:}" failed. No retries permitted until 2026-07-28 19:48:05.025653579 +0000 UTC m=+66.206170612 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs") pod "network-metrics-daemon-vwvtz" (UID: "6c6e1a80-f190-4d1d-9e27-a00b79bee30c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:47:33.025822 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:33.025689    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret podName:4028a654-3327-4719-b4d0-6434b1573bbd nodeName:}" failed. No retries permitted until 2026-07-28 19:48:05.025681088 +0000 UTC m=+66.206198122 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret") pod "global-pull-secret-syncer-hcb5b" (UID: "4028a654-3327-4719-b4d0-6434b1573bbd") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:47:33.360459 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:33.360382    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:33.360617 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:33.360382    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:33.360617 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:33.360508    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:33.360617 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:33.360382    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:33.360617 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:33.360594    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:33.360769 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:33.360655    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:34.550956 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:34.550890    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:34.651737 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:34.651712    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:34.761155 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:34.761131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:34.858233 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:34.858148    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:34.970800 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:34.970771    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:35.239815 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:35.239789    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:35.360254 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:35.360233    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:35.360254 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:35.360245    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:35.360446 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:35.360267    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:35.360446 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:35.360394    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:35.360552 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:35.360514    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:35.360608 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:35.360587    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:35.528922 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:35.528838    2569 generic.go:356] "Generic (PLEG): container finished" podID="8893a3f0-c25c-4f4b-906b-be6e0cf425c9" containerID="5167599d79ff13e68cfc37606c723b3676a0d236640b538649bf92dc05de5f62" exitCode=0
Jul 28 19:47:35.528922 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:35.528901    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hqwp" event={"ID":"8893a3f0-c25c-4f4b-906b-be6e0cf425c9","Type":"ContainerDied","Data":"5167599d79ff13e68cfc37606c723b3676a0d236640b538649bf92dc05de5f62"}
Jul 28 19:47:36.237847 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:36.237803    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:36.534153 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:36.534066    2569 generic.go:356] "Generic (PLEG): container finished" podID="8893a3f0-c25c-4f4b-906b-be6e0cf425c9" containerID="7fc8550aa60b4b5b0929e0a2999570e3e64490210dca1c674e4e145e939eaf79" exitCode=0
Jul 28 19:47:36.534153 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:36.534121    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hqwp" event={"ID":"8893a3f0-c25c-4f4b-906b-be6e0cf425c9","Type":"ContainerDied","Data":"7fc8550aa60b4b5b0929e0a2999570e3e64490210dca1c674e4e145e939eaf79"}
Jul 28 19:47:36.637666 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:36.637643    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:36.866091 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:36.866038    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:36.924980 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:36.924948    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:37.240974 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:37.240947    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:37.360045 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:37.360020    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:37.360173 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:37.360020    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:37.360173 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:37.360137    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:37.360291 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:37.360254    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:37.360341 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:37.360306    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:37.360461 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:37.360441    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:37.396444 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:37.396419    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:37.539057 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:37.539001    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hqwp" event={"ID":"8893a3f0-c25c-4f4b-906b-be6e0cf425c9","Type":"ContainerStarted","Data":"170826cd17436169a29a94e562eb4e298db8832f92ec619cebf4463e7d56a4ac"}
Jul 28 19:47:37.551038 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:37.551018    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:37.558637 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:37.558599    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2hqwp" podStartSLOduration=9.784845495 podStartE2EDuration="38.558587063s" totalImagesPullingTime="28.773741568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:37.557621046 +0000 UTC m=+38.738138102" watchObservedRunningTime="2026-07-28 19:47:37.558587063 +0000 UTC m=+38.739104119"
Jul 28 19:47:38.038154 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:38.038125    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:38.212228 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:38.212201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:38.267860 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:38.267835    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:38.628611 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:38.628584    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:38.839660 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:38.839633    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:38.939779 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:38.939748    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:39.239544 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:39.239479    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:39.361445 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:39.361419    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:39.361747 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:39.361505    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:39.361747 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:39.361612    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:39.361747 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:39.361730    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:39.361854 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:39.361788    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:39.361886 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:39.361861    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:39.635940 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:39.635877    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:39.839290 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:39.839251    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:40.445316 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:40.445269    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:40.640593 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:40.640569    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:40.925054 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:40.925017    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:41.248151 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:41.248084    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:41.360208 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:41.360181    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:41.360208 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:41.360198    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:41.360422 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:41.360274    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:41.360422 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:41.360320    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:41.360422 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:41.360396    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:41.360581 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:41.360483    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:41.639701 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:41.639644    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:42.041842 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:42.041816    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:42.212504 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:42.212476    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:42.312946 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:42.312886    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:42.360369 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:42.360343    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:42.443316 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:42.443272    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:42.509507 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:42.509483    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:42.678038 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:42.678014    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:42.729449 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:42.729423    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:42.839962 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:42.839940    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:43.041538 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:43.041480    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:43.360345 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:43.360269    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:43.360345 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:43.360309    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:43.360493 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:43.360396    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:43.360493 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:43.360474    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:43.360561 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:43.360514    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:43.360619 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:43.360598    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:43.441459 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:43.441436    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:43.542291 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:43.542257    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:43.617815 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:43.617761    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:43.949609 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:43.949580    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:43.998140 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:43.998114    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:44.099678 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:44.099657    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:44.257035 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:44.256974    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:44.356101 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:44.356076    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:44.455975 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:44.455956    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:44.838166 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:44.838137    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:45.354568 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:45.354537    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:45.360584 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:45.360567    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:45.360672 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:45.360582    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:45.360672 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:45.360662    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:45.360756 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:45.360728    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:45.360798 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:45.360762    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:45.360857 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:45.360842    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:45.457409 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:45.457385    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:45.638213 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:45.638142    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:45.782223 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:45.782195    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:45.883088 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:45.883062    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:46.120663 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:46.120634    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:47.360503 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:47.360468    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:47.360503 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:47.360491    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:47.360927 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:47.360499    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:47.360927 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:47.360562    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:47.360927 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:47.360665    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:47.360927 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:47.360742    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:49.360382 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:49.360351    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:49.360879 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:49.360481    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:49.360879 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:49.360553    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:49.360879 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:49.360650    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:49.360879 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:49.360680    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:49.360879 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:49.360763    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:51.359987 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:51.359954    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:51.359987 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:51.359976    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:51.360517 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:51.359959    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:51.360517 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:51.360084    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:51.360517 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:51.360182    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:51.360517 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:51.360242    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:52.937533 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:52.937504    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:53.019777 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:53.019750    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:53.136078 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:53.136053    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:53.220983 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:53.220925    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:53.292694 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:53.292663    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:53.359748 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:53.359726    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:53.359748 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:53.359736    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:53.359898 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:53.359752    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:53.359898 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:53.359850    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vwvtz" podUID="6c6e1a80-f190-4d1d-9e27-a00b79bee30c"
Jul 28 19:47:53.360006 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:53.359918    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hcb5b" podUID="4028a654-3327-4719-b4d0-6434b1573bbd"
Jul 28 19:47:53.360062 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:47:53.359994    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcl5s" podUID="8a92e62e-b631-4596-8efc-b4e65ff8e290"
Jul 28 19:47:54.185510 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.185490    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-138.ec2.internal" event="NodeReady"
Jul 28 19:47:54.185863 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.185580    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 19:47:54.230991 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.230953    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xlrkr"]
Jul 28 19:47:54.232880 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.232863    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nwvlj"]
Jul 28 19:47:54.232994 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.232974    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.235341 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.235315    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 19:47:54.235609 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.235590    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 19:47:54.236898 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.236175    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wfpzj\""
Jul 28 19:47:54.237034 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.237000    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nwvlj"
Jul 28 19:47:54.238941 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.238925    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 19:47:54.239152 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.239133    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 19:47:54.239231 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.239139    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zg9s9\""
Jul 28 19:47:54.239317 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.239230    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 19:47:54.242555 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.242535    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nwvlj"]
Jul 28 19:47:54.245507 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.245485    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xlrkr"]
Jul 28 19:47:54.332585 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.332565    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nt9qg"]
Jul 28 19:47:54.334310 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.334271    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.336100 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.336082    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 19:47:54.336185 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.336107    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t6cqg\""
Jul 28 19:47:54.336185 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.336136    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 19:47:54.336272 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.336239    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 19:47:54.336412 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.336393    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 19:47:54.342831 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.342811    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nt9qg"]
Jul 28 19:47:54.351395 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.351374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:54.386201 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.386178    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpq6h\" (UniqueName: \"kubernetes.io/projected/838e134c-af5c-4c02-bfc5-886832884a5c-kube-api-access-lpq6h\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.386313 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.386214    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/838e134c-af5c-4c02-bfc5-886832884a5c-config-volume\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.386313 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.386237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65b4t\" (UniqueName: \"kubernetes.io/projected/f3fd2311-f24e-41a4-a01e-f6f388dec0ae-kube-api-access-65b4t\") pod \"ingress-canary-nwvlj\" (UID: \"f3fd2311-f24e-41a4-a01e-f6f388dec0ae\") " pod="openshift-ingress-canary/ingress-canary-nwvlj"
Jul 28 19:47:54.386313 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.386274    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/838e134c-af5c-4c02-bfc5-886832884a5c-metrics-tls\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.386313 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.386307    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3fd2311-f24e-41a4-a01e-f6f388dec0ae-cert\") pod \"ingress-canary-nwvlj\" (UID: \"f3fd2311-f24e-41a4-a01e-f6f388dec0ae\") " pod="openshift-ingress-canary/ingress-canary-nwvlj"
Jul 28 19:47:54.386465 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.386335    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/838e134c-af5c-4c02-bfc5-886832884a5c-tmp-dir\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.402084 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.402061    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:54.487168 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487118    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpq6h\" (UniqueName: \"kubernetes.io/projected/838e134c-af5c-4c02-bfc5-886832884a5c-kube-api-access-lpq6h\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.487168 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487160    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/838e134c-af5c-4c02-bfc5-886832884a5c-config-volume\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.487269 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65b4t\" (UniqueName: \"kubernetes.io/projected/f3fd2311-f24e-41a4-a01e-f6f388dec0ae-kube-api-access-65b4t\") pod \"ingress-canary-nwvlj\" (UID: \"f3fd2311-f24e-41a4-a01e-f6f388dec0ae\") " pod="openshift-ingress-canary/ingress-canary-nwvlj"
Jul 28 19:47:54.487269 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487207    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58e5525f-07b2-4562-b3bd-452985cbba23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.487269 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58e5525f-07b2-4562-b3bd-452985cbba23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.487269 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/838e134c-af5c-4c02-bfc5-886832884a5c-metrics-tls\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.487466 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487272    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/58e5525f-07b2-4562-b3bd-452985cbba23-data-volume\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.487466 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3fd2311-f24e-41a4-a01e-f6f388dec0ae-cert\") pod \"ingress-canary-nwvlj\" (UID: \"f3fd2311-f24e-41a4-a01e-f6f388dec0ae\") " pod="openshift-ingress-canary/ingress-canary-nwvlj"
Jul 28 19:47:54.487466 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487367    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/58e5525f-07b2-4562-b3bd-452985cbba23-crio-socket\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.487466 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487417    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/838e134c-af5c-4c02-bfc5-886832884a5c-tmp-dir\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.487466 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487457    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm74z\" (UniqueName: \"kubernetes.io/projected/58e5525f-07b2-4562-b3bd-452985cbba23-kube-api-access-dm74z\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.487698 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487664    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/838e134c-af5c-4c02-bfc5-886832884a5c-tmp-dir\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.487737 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.487712    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/838e134c-af5c-4c02-bfc5-886832884a5c-config-volume\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.491171 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.491142    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/838e134c-af5c-4c02-bfc5-886832884a5c-metrics-tls\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.491251 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.491187    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3fd2311-f24e-41a4-a01e-f6f388dec0ae-cert\") pod \"ingress-canary-nwvlj\" (UID: \"f3fd2311-f24e-41a4-a01e-f6f388dec0ae\") " pod="openshift-ingress-canary/ingress-canary-nwvlj"
Jul 28 19:47:54.493937 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.493914    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpq6h\" (UniqueName: \"kubernetes.io/projected/838e134c-af5c-4c02-bfc5-886832884a5c-kube-api-access-lpq6h\") pod \"dns-default-xlrkr\" (UID: \"838e134c-af5c-4c02-bfc5-886832884a5c\") " pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.494118 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.494101    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65b4t\" (UniqueName: \"kubernetes.io/projected/f3fd2311-f24e-41a4-a01e-f6f388dec0ae-kube-api-access-65b4t\") pod \"ingress-canary-nwvlj\" (UID: \"f3fd2311-f24e-41a4-a01e-f6f388dec0ae\") " pod="openshift-ingress-canary/ingress-canary-nwvlj"
Jul 28 19:47:54.546971 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.546946    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:47:54.552158 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.552132    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nwvlj"
Jul 28 19:47:54.588847 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.587842    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58e5525f-07b2-4562-b3bd-452985cbba23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.588847 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.587888    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58e5525f-07b2-4562-b3bd-452985cbba23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.588847 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.587938    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/58e5525f-07b2-4562-b3bd-452985cbba23-data-volume\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.588847 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.587971    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/58e5525f-07b2-4562-b3bd-452985cbba23-crio-socket\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.588847 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.588019    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm74z\" (UniqueName: \"kubernetes.io/projected/58e5525f-07b2-4562-b3bd-452985cbba23-kube-api-access-dm74z\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.589215 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.589138    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/58e5525f-07b2-4562-b3bd-452985cbba23-data-volume\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.589335 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.589313    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/58e5525f-07b2-4562-b3bd-452985cbba23-crio-socket\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.589419 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.589403    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58e5525f-07b2-4562-b3bd-452985cbba23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.591555 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.591522    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58e5525f-07b2-4562-b3bd-452985cbba23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.596477 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.596454    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm74z\" (UniqueName: \"kubernetes.io/projected/58e5525f-07b2-4562-b3bd-452985cbba23-kube-api-access-dm74z\") pod \"insights-runtime-extractor-nt9qg\" (UID: \"58e5525f-07b2-4562-b3bd-452985cbba23\") " pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.642642 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.642573    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nt9qg"
Jul 28 19:47:54.682770 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.682739    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nwvlj"]
Jul 28 19:47:54.687953 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:54.687923    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3fd2311_f24e_41a4_a01e_f6f388dec0ae.slice/crio-e361fd6ae00280971e8c1ef9f4894fa09ffc1221b201907868511ff812f17463 WatchSource:0}: Error finding container e361fd6ae00280971e8c1ef9f4894fa09ffc1221b201907868511ff812f17463: Status 404 returned error can't find the container with id e361fd6ae00280971e8c1ef9f4894fa09ffc1221b201907868511ff812f17463
Jul 28 19:47:54.695919 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.695885    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xlrkr"]
Jul 28 19:47:54.702794 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:54.702759    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod838e134c_af5c_4c02_bfc5_886832884a5c.slice/crio-7ac8d12aeb0d895efccba958893339080a8a83384293999bd19ce81f85f82d47 WatchSource:0}: Error finding container 7ac8d12aeb0d895efccba958893339080a8a83384293999bd19ce81f85f82d47: Status 404 returned error can't find the container with id 7ac8d12aeb0d895efccba958893339080a8a83384293999bd19ce81f85f82d47
Jul 28 19:47:54.775596 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:54.775529    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nt9qg"]
Jul 28 19:47:54.778892 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:54.778867    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58e5525f_07b2_4562_b3bd_452985cbba23.slice/crio-2c263a2a79ae9c1be1e6146573280db99dc55e6c14f0ccbf1cc54c1dd55a4628 WatchSource:0}: Error finding container 2c263a2a79ae9c1be1e6146573280db99dc55e6c14f0ccbf1cc54c1dd55a4628: Status 404 returned error can't find the container with id 2c263a2a79ae9c1be1e6146573280db99dc55e6c14f0ccbf1cc54c1dd55a4628
Jul 28 19:47:55.359735 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.359699    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:47:55.360188 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.359700    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:47:55.360188 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.360013    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:47:55.362576 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.362348    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lhtrs\""
Jul 28 19:47:55.362576 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.362363    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l4wgt\""
Jul 28 19:47:55.362576 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.362393    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:47:55.362576 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.362352    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:47:55.362896 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.362692    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:47:55.362896 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.362835    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:47:55.580325 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.580272    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nt9qg" event={"ID":"58e5525f-07b2-4562-b3bd-452985cbba23","Type":"ContainerStarted","Data":"2c263a2a79ae9c1be1e6146573280db99dc55e6c14f0ccbf1cc54c1dd55a4628"}
Jul 28 19:47:55.581784 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.581753    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nwvlj" event={"ID":"f3fd2311-f24e-41a4-a01e-f6f388dec0ae","Type":"ContainerStarted","Data":"e361fd6ae00280971e8c1ef9f4894fa09ffc1221b201907868511ff812f17463"}
Jul 28 19:47:55.583217 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.583165    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xlrkr" event={"ID":"838e134c-af5c-4c02-bfc5-886832884a5c","Type":"ContainerStarted","Data":"7ac8d12aeb0d895efccba958893339080a8a83384293999bd19ce81f85f82d47"}
Jul 28 19:47:55.706719 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.706660    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:55.808725 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:55.808687    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:56.956187 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:56.956143    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:57.057820 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.057772    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:57.117682 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.117648    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg"]
Jul 28 19:47:57.119728 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.119693    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg"
Jul 28 19:47:57.121561 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.121536    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 19:47:57.121682 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.121545    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hcdw8\""
Jul 28 19:47:57.126926 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.126871    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg"]
Jul 28 19:47:57.130515 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.130476    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:57.207627 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.207512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:57.215654 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.215629    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c6ed8149-d7bf-49b0-b7b2-f165f1ebd63d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9rcqg\" (UID: \"c6ed8149-d7bf-49b0-b7b2-f165f1ebd63d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg"
Jul 28 19:47:57.308431 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.308401    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:57.316433 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.316394    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c6ed8149-d7bf-49b0-b7b2-f165f1ebd63d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9rcqg\" (UID: \"c6ed8149-d7bf-49b0-b7b2-f165f1ebd63d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg"
Jul 28 19:47:57.319526 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.319503    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c6ed8149-d7bf-49b0-b7b2-f165f1ebd63d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9rcqg\" (UID: \"c6ed8149-d7bf-49b0-b7b2-f165f1ebd63d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg"
Jul 28 19:47:57.408860 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.408827    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:57.432025 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.432001    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg"
Jul 28 19:47:57.709724 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.709689    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg"]
Jul 28 19:47:57.924109 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.924081    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:57.974531 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:57.974498    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:58.592384 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:58.592345    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xlrkr" event={"ID":"838e134c-af5c-4c02-bfc5-886832884a5c","Type":"ContainerStarted","Data":"72d2f1585f90411b55ee931e537c778aa2402af7b0b0516203efe73fdba2c74a"}
Jul 28 19:47:58.593911 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:58.593877    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nwvlj" event={"ID":"f3fd2311-f24e-41a4-a01e-f6f388dec0ae","Type":"ContainerStarted","Data":"546a751d73efb0ef08524b826a7c76223c93dfdc8523c67d3dee89b9a06f75a5"}
Jul 28 19:47:58.607821 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:58.607771    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nwvlj" podStartSLOduration=1.7134617250000002 podStartE2EDuration="4.607750584s" totalImagesPullingTime="2.894288859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:47:58.606842657 +0000 UTC m=+59.787359716" watchObservedRunningTime="2026-07-28 19:47:58.607750584 +0000 UTC m=+59.788267624"
Jul 28 19:47:59.524461 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:59.524430    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5m4m9"
Jul 28 19:47:59.637852 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:59.637814    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:59.693371 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:59.693340    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:59.742817 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:47:59.742795    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:59.931603 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:47:59.931576    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ed8149_d7bf_49b0_b7b2_f165f1ebd63d.slice/crio-a1301067ae2bb2b5c1b19f12a48df59b0c24bf898312333bfa2e2c379d2a6028 WatchSource:0}: Error finding container a1301067ae2bb2b5c1b19f12a48df59b0c24bf898312333bfa2e2c379d2a6028: Status 404 returned error can't find the container with id a1301067ae2bb2b5c1b19f12a48df59b0c24bf898312333bfa2e2c379d2a6028
Jul 28 19:48:00.009733 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:00.009701    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:00.064386 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:00.064363    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:00.124029 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:00.124000    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:00.225649 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:00.225566    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:00.603023 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:00.602922    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg" event={"ID":"c6ed8149-d7bf-49b0-b7b2-f165f1ebd63d","Type":"ContainerStarted","Data":"a1301067ae2bb2b5c1b19f12a48df59b0c24bf898312333bfa2e2c379d2a6028"}
Jul 28 19:48:00.721895 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:00.721860    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:00.822685 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:00.822638    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:00.875526 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:00.875441    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:01.503780 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.503747    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66cb8cc574-4vcwl"]
Jul 28 19:48:01.505742 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.505716    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.507787 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.507765    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 19:48:01.508019 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.507993    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lxsh7\""
Jul 28 19:48:01.508142 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.508002    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 19:48:01.508497 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.508477    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 19:48:01.508620 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.508517    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 19:48:01.508620 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.508582    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 19:48:01.508746 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.508718    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 19:48:01.508864 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.508847    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 19:48:01.514170 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.514017    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 19:48:01.517502 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.517430    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66cb8cc574-4vcwl"]
Jul 28 19:48:01.546030 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.545999    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-config\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.546186 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.546039    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-oauth-serving-cert\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.546186 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.546094    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-service-ca\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.546186 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.546129    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-oauth-config\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.546186 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.546155    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-trusted-ca-bundle\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.546418 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.546256    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-serving-cert\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.546418 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.546322    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm94h\" (UniqueName: \"kubernetes.io/projected/503ff851-95fa-47c8-85e4-0fccec1dcba1-kube-api-access-sm94h\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.608828 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.608739    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nt9qg" event={"ID":"58e5525f-07b2-4562-b3bd-452985cbba23","Type":"ContainerStarted","Data":"5f5da50d7e0721cb050d5830366e77e8581fbd41aff87c34dab09cad50572d8d"}
Jul 28 19:48:01.610442 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.610409    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg" event={"ID":"c6ed8149-d7bf-49b0-b7b2-f165f1ebd63d","Type":"ContainerStarted","Data":"898c434b06f132978874b626bdad9e0f50623f7617bab599570099ffaa9d55fc"}
Jul 28 19:48:01.610827 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.610805    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg"
Jul 28 19:48:01.612848 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.612818    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xlrkr" event={"ID":"838e134c-af5c-4c02-bfc5-886832884a5c","Type":"ContainerStarted","Data":"f0e344b885868e15e13f3328c2ec260b64aea9e8c1e283bb20e1acb3fd8ff3e8"}
Jul 28 19:48:01.613032 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.613008    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:48:01.616666 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.616634    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg"
Jul 28 19:48:01.624530 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.624235    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9rcqg" podStartSLOduration=3.226485552 podStartE2EDuration="4.624216531s" totalImagesPullingTime="1.397730979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:01.624005768 +0000 UTC m=+62.804522824" watchObservedRunningTime="2026-07-28 19:48:01.624216531 +0000 UTC m=+62.804733588"
Jul 28 19:48:01.646751 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.646688    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-trusted-ca-bundle\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.646909 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.646832    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-serving-cert\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.646909 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.646874    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm94h\" (UniqueName: \"kubernetes.io/projected/503ff851-95fa-47c8-85e4-0fccec1dcba1-kube-api-access-sm94h\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.646994 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.646964    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-config\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.647035 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.646994    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-oauth-serving-cert\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.647067 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.647033    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-service-ca\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.647067 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.647061    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-oauth-config\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.648256 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.648228    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-config\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.648373 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.648315    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-oauth-serving-cert\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.648859 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.648835    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-service-ca\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.649260 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.649237    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-trusted-ca-bundle\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.651047 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.651025    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-serving-cert\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.653337 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.653314    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-oauth-config\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.660635 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.660592    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm94h\" (UniqueName: \"kubernetes.io/projected/503ff851-95fa-47c8-85e4-0fccec1dcba1-kube-api-access-sm94h\") pod \"console-66cb8cc574-4vcwl\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") " pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.818493 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.818448    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:01.965590 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.965529    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xlrkr" podStartSLOduration=5.09248858 podStartE2EDuration="7.965504771s" totalImagesPullingTime="2.873016191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:01.659550318 +0000 UTC m=+62.840067354" watchObservedRunningTime="2026-07-28 19:48:01.965504771 +0000 UTC m=+63.146021830"
Jul 28 19:48:01.965949 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:01.965926    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66cb8cc574-4vcwl"]
Jul 28 19:48:01.969290 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:01.969244    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503ff851_95fa_47c8_85e4_0fccec1dcba1.slice/crio-4b3d8a96cb9d2d4c820cf0800bee4203085fd93749ce086ddef4aebadcecfb72 WatchSource:0}: Error finding container 4b3d8a96cb9d2d4c820cf0800bee4203085fd93749ce086ddef4aebadcecfb72: Status 404 returned error can't find the container with id 4b3d8a96cb9d2d4c820cf0800bee4203085fd93749ce086ddef4aebadcecfb72
Jul 28 19:48:02.143067 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.142988    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:02.179507 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.179478    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"]
Jul 28 19:48:02.181541 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.181523    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.183773 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.183522    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 28 19:48:02.183773 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.183549    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 28 19:48:02.183939 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.183818    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 19:48:02.183939 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.183866    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 19:48:02.184087 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.184072    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 19:48:02.184205 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.184187    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-4w9h5\""
Jul 28 19:48:02.189853 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.189744    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"]
Jul 28 19:48:02.217091 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.217066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:02.252974 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.252950    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/49041c8a-764a-41ee-aa27-96ff41625896-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.253083 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.253010    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49041c8a-764a-41ee-aa27-96ff41625896-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.253127 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.253097    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z7lf\" (UniqueName: \"kubernetes.io/projected/49041c8a-764a-41ee-aa27-96ff41625896-kube-api-access-6z7lf\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.253169 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.253146    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49041c8a-764a-41ee-aa27-96ff41625896-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.354158 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.354123    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z7lf\" (UniqueName: \"kubernetes.io/projected/49041c8a-764a-41ee-aa27-96ff41625896-kube-api-access-6z7lf\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.354319 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.354177    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49041c8a-764a-41ee-aa27-96ff41625896-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.354319 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.354220    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/49041c8a-764a-41ee-aa27-96ff41625896-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.354319 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.354270    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49041c8a-764a-41ee-aa27-96ff41625896-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.354487 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:48:02.354405    2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 28 19:48:02.354487 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:48:02.354480    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49041c8a-764a-41ee-aa27-96ff41625896-prometheus-operator-tls podName:49041c8a-764a-41ee-aa27-96ff41625896 nodeName:}" failed. No retries permitted until 2026-07-28 19:48:02.854458932 +0000 UTC m=+64.034975979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/49041c8a-764a-41ee-aa27-96ff41625896-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-j2qdz" (UID: "49041c8a-764a-41ee-aa27-96ff41625896") : secret "prometheus-operator-tls" not found
Jul 28 19:48:02.355085 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.355058    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49041c8a-764a-41ee-aa27-96ff41625896-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.356752 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.356732    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49041c8a-764a-41ee-aa27-96ff41625896-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.358226 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.358198    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:02.361844 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.361802    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z7lf\" (UniqueName: \"kubernetes.io/projected/49041c8a-764a-41ee-aa27-96ff41625896-kube-api-access-6z7lf\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.458958 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.458913    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:02.618780 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.618740    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66cb8cc574-4vcwl" event={"ID":"503ff851-95fa-47c8-85e4-0fccec1dcba1","Type":"ContainerStarted","Data":"4b3d8a96cb9d2d4c820cf0800bee4203085fd93749ce086ddef4aebadcecfb72"}
Jul 28 19:48:02.714071 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.714000    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:02.763300 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.763261    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:02.858628 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.858589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/49041c8a-764a-41ee-aa27-96ff41625896-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:02.861407 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:02.861379    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/49041c8a-764a-41ee-aa27-96ff41625896-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-j2qdz\" (UID: \"49041c8a-764a-41ee-aa27-96ff41625896\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:03.092740 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:03.092649    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"
Jul 28 19:48:03.615902 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:03.615868    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:03.717432 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:03.717400    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:04.592952 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:04.592920    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-j2qdz"]
Jul 28 19:48:04.596933 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:04.596904    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49041c8a_764a_41ee_aa27_96ff41625896.slice/crio-0dc5f627faf345f89ff1b0761291a9a10ab4ed48796962a94a9d77923e90fa64 WatchSource:0}: Error finding container 0dc5f627faf345f89ff1b0761291a9a10ab4ed48796962a94a9d77923e90fa64: Status 404 returned error can't find the container with id 0dc5f627faf345f89ff1b0761291a9a10ab4ed48796962a94a9d77923e90fa64
Jul 28 19:48:04.631916 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:04.631884    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz" event={"ID":"49041c8a-764a-41ee-aa27-96ff41625896","Type":"ContainerStarted","Data":"0dc5f627faf345f89ff1b0761291a9a10ab4ed48796962a94a9d77923e90fa64"}
Jul 28 19:48:04.634253 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:04.634192    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nt9qg" event={"ID":"58e5525f-07b2-4562-b3bd-452985cbba23","Type":"ContainerStarted","Data":"1fcce94d5e1c6e3ca562ead2d68e8001765eea81cbf153fa59af4bb531a1b457"}
Jul 28 19:48:04.980010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:04.979968    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4fq7\" (UniqueName: \"kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7\") pod \"network-check-target-qcl5s\" (UID: \"8a92e62e-b631-4596-8efc-b4e65ff8e290\") " pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:48:04.981941 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:04.981917    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:48:04.992441 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:04.992417    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:48:05.003641 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.003592    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4fq7\" (UniqueName: \"kubernetes.io/projected/8a92e62e-b631-4596-8efc-b4e65ff8e290-kube-api-access-q4fq7\") pod \"network-check-target-qcl5s\" (UID: \"8a92e62e-b631-4596-8efc-b4e65ff8e290\") " pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:48:05.080478 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.080449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:48:05.080603 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.080494    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:48:05.082443 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.082421    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:48:05.082443 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.082421    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:48:05.093400 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.093367    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4028a654-3327-4719-b4d0-6434b1573bbd-original-pull-secret\") pod \"global-pull-secret-syncer-hcb5b\" (UID: \"4028a654-3327-4719-b4d0-6434b1573bbd\") " pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:48:05.093563 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.093433    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c6e1a80-f190-4d1d-9e27-a00b79bee30c-metrics-certs\") pod \"network-metrics-daemon-vwvtz\" (UID: \"6c6e1a80-f190-4d1d-9e27-a00b79bee30c\") " pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:48:05.276228 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.276148    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lhtrs\""
Jul 28 19:48:05.281996 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.281968    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hcb5b"
Jul 28 19:48:05.284237 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.284069    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:48:05.290648 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.290615    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l4wgt\""
Jul 28 19:48:05.299233 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:05.299200    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vwvtz"
Jul 28 19:48:06.110140 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:06.110100    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:06.211402 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:06.211345    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:06.563820 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:06.563790    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qcl5s"]
Jul 28 19:48:06.567695 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:06.567662    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a92e62e_b631_4596_8efc_b4e65ff8e290.slice/crio-b70983667d51e82198c02bb7e94104582ba5b25244430ff14cb0ecea90b057d7 WatchSource:0}: Error finding container b70983667d51e82198c02bb7e94104582ba5b25244430ff14cb0ecea90b057d7: Status 404 returned error can't find the container with id b70983667d51e82198c02bb7e94104582ba5b25244430ff14cb0ecea90b057d7
Jul 28 19:48:06.642216 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:06.642188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qcl5s" event={"ID":"8a92e62e-b631-4596-8efc-b4e65ff8e290","Type":"ContainerStarted","Data":"b70983667d51e82198c02bb7e94104582ba5b25244430ff14cb0ecea90b057d7"}
Jul 28 19:48:06.784546 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:06.784480    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vwvtz"]
Jul 28 19:48:06.787333 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:06.787266    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hcb5b"]
Jul 28 19:48:07.253455 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:07.253425    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:07.312423 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:07.312395    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:08.471989 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:08.471950    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4028a654_3327_4719_b4d0_6434b1573bbd.slice/crio-46d6711ec19a9ff932e39f8e5aa06a782305f330a9d77ba8862a1c9b6040bee7 WatchSource:0}: Error finding container 46d6711ec19a9ff932e39f8e5aa06a782305f330a9d77ba8862a1c9b6040bee7: Status 404 returned error can't find the container with id 46d6711ec19a9ff932e39f8e5aa06a782305f330a9d77ba8862a1c9b6040bee7
Jul 28 19:48:08.472855 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:08.472580    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c6e1a80_f190_4d1d_9e27_a00b79bee30c.slice/crio-d4bc5587998973289cbdbad05a90b553512932fb4037d7aec1ec3504fae0d1fd WatchSource:0}: Error finding container d4bc5587998973289cbdbad05a90b553512932fb4037d7aec1ec3504fae0d1fd: Status 404 returned error can't find the container with id d4bc5587998973289cbdbad05a90b553512932fb4037d7aec1ec3504fae0d1fd
Jul 28 19:48:08.621807 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:08.621767    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xlrkr"
Jul 28 19:48:08.652224 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:08.652123    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hcb5b" event={"ID":"4028a654-3327-4719-b4d0-6434b1573bbd","Type":"ContainerStarted","Data":"46d6711ec19a9ff932e39f8e5aa06a782305f330a9d77ba8862a1c9b6040bee7"}
Jul 28 19:48:08.654319 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:08.654272    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vwvtz" event={"ID":"6c6e1a80-f190-4d1d-9e27-a00b79bee30c","Type":"ContainerStarted","Data":"d4bc5587998973289cbdbad05a90b553512932fb4037d7aec1ec3504fae0d1fd"}
Jul 28 19:48:09.091137 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:09.091096    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:09.145224 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:09.145074    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:09.200267 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:09.200196    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:09.660655 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:09.660154    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz" event={"ID":"49041c8a-764a-41ee-aa27-96ff41625896","Type":"ContainerStarted","Data":"2091ce77f33e77d906b358b7cc0f981fde19425fedff4deeda0c37d9f53ff3e5"}
Jul 28 19:48:09.660655 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:09.660212    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz" event={"ID":"49041c8a-764a-41ee-aa27-96ff41625896","Type":"ContainerStarted","Data":"ef4313b818d0216fd114a8433f4b305ff152b716bd5fb0df631835dede1284a1"}
Jul 28 19:48:09.662717 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:09.662665    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66cb8cc574-4vcwl" event={"ID":"503ff851-95fa-47c8-85e4-0fccec1dcba1","Type":"ContainerStarted","Data":"e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1"}
Jul 28 19:48:09.665311 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:09.665265    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nt9qg" event={"ID":"58e5525f-07b2-4562-b3bd-452985cbba23","Type":"ContainerStarted","Data":"04074f8b80f86f281903b1bf0524bf4877db56b13acd43ed05c58cd3b7f75008"}
Jul 28 19:48:09.675714 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:09.675666    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-j2qdz" podStartSLOduration=3.409830346 podStartE2EDuration="7.675648294s" totalImagesPullingTime="4.265817948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:09.674873885 +0000 UTC m=+70.855390942" watchObservedRunningTime="2026-07-28 19:48:09.675648294 +0000 UTC m=+70.856165348"
Jul 28 19:48:09.690119 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:09.689878    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nt9qg" podStartSLOduration=1.710442969 podStartE2EDuration="15.689860304s" totalImagesPullingTime="13.979417335s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:47:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:09.688858592 +0000 UTC m=+70.869375649" watchObservedRunningTime="2026-07-28 19:48:09.689860304 +0000 UTC m=+70.870377361"
Jul 28 19:48:10.671835 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:10.671797    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vwvtz" event={"ID":"6c6e1a80-f190-4d1d-9e27-a00b79bee30c","Type":"ContainerStarted","Data":"3ecf0356f3c94c0869a0c3606fea69eaaab6eb4f0b9b69858b7d7df237c82f6f"}
Jul 28 19:48:10.672457 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:10.671864    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vwvtz" event={"ID":"6c6e1a80-f190-4d1d-9e27-a00b79bee30c","Type":"ContainerStarted","Data":"4c7c58cd3ea3024b6fb0128573d93809d4306833a0176f4b8217ae6df6d6e2fa"}
Jul 28 19:48:10.686033 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:10.685968    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66cb8cc574-4vcwl" podStartSLOduration=2.799492289 podStartE2EDuration="9.685946357s" totalImagesPullingTime="6.886454068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:48:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:09.705936348 +0000 UTC m=+70.886453420" watchObservedRunningTime="2026-07-28 19:48:10.685946357 +0000 UTC m=+71.866463417"
Jul 28 19:48:10.686671 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:10.686632    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vwvtz" podStartSLOduration=70.573159127 podStartE2EDuration="1m11.686619913s" totalImagesPullingTime="1.113460786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:10.684660913 +0000 UTC m=+71.865177968" watchObservedRunningTime="2026-07-28 19:48:10.686619913 +0000 UTC m=+71.867136971"
Jul 28 19:48:10.790895 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:10.790819    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:10.887513 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:10.887482    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:11.289235 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.289205    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:11.501027 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.500986    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:11.525558 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.525528    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"]
Jul 28 19:48:11.527987 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.527963    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.529971 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.529948    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 19:48:11.530108 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.530062    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 19:48:11.530185 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.530116    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xkkkv\""
Jul 28 19:48:11.537909 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.537886    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"]
Jul 28 19:48:11.541555 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.541503    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"]
Jul 28 19:48:11.544350 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.544329    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.546525 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.546493    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 19:48:11.548894 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.547227    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 19:48:11.548894 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.547454    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 19:48:11.548894 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.547811    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-wj9vv\""
Jul 28 19:48:11.548894 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.548263    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lbz5l"]
Jul 28 19:48:11.554603 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.553058    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.555451 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.555426    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lrswp\""
Jul 28 19:48:11.555553 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.555428    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 19:48:11.555794 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.555773    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 19:48:11.556525 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.556505    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 19:48:11.558476 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.558451    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"]
Jul 28 19:48:11.589511 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.589461    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:11.635846 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.635817    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.635966 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.635858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-wtmp\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.635966 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.635894    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a983224-41c0-4cb5-974b-bc589225ca41-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.635966 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.635925    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx2sq\" (UniqueName: \"kubernetes.io/projected/4a983224-41c0-4cb5-974b-bc589225ca41-kube-api-access-qx2sq\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.635966 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.635957    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.636189 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.635980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.636189 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.635996    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6149ebba-f436-44a9-851d-b9e6f4c543bf-metrics-client-ca\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.636189 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636027    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a983224-41c0-4cb5-974b-bc589225ca41-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.636189 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636047    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6149ebba-f436-44a9-851d-b9e6f4c543bf-sys\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.636189 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636075    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/973f742f-b3da-44ee-b805-af2d1db8fdf0-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.636189 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6149ebba-f436-44a9-851d-b9e6f4c543bf-root\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.636189 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-textfile\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.636189 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636155    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.636189 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636169    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7dzx\" (UniqueName: \"kubernetes.io/projected/6149ebba-f436-44a9-851d-b9e6f4c543bf-kube-api-access-g7dzx\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.636662 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.636662 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636292    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnp7k\" (UniqueName: \"kubernetes.io/projected/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-api-access-lnp7k\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.636662 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636322    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-tls\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.636662 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636344    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/973f742f-b3da-44ee-b805-af2d1db8fdf0-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.636662 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.636370    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a983224-41c0-4cb5-974b-bc589225ca41-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.653749 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.653725    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:11.737677 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.737651    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a983224-41c0-4cb5-974b-bc589225ca41-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.738070 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.737710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx2sq\" (UniqueName: \"kubernetes.io/projected/4a983224-41c0-4cb5-974b-bc589225ca41-kube-api-access-qx2sq\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.738070 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.737751    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.738070 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.737779    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.738070 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.737801    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6149ebba-f436-44a9-851d-b9e6f4c543bf-metrics-client-ca\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.738070 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.737821    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a983224-41c0-4cb5-974b-bc589225ca41-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.738070 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.737847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6149ebba-f436-44a9-851d-b9e6f4c543bf-sys\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.738070 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:48:11.737894    2569 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 28 19:48:11.738070 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:48:11.737960    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-tls podName:973f742f-b3da-44ee-b805-af2d1db8fdf0 nodeName:}" failed. No retries permitted until 2026-07-28 19:48:12.237939008 +0000 UTC m=+73.418456054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-956jc" (UID: "973f742f-b3da-44ee-b805-af2d1db8fdf0") : secret "kube-state-metrics-tls" not found
Jul 28 19:48:11.738070 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.737895    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/973f742f-b3da-44ee-b805-af2d1db8fdf0-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.738070 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738055    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6149ebba-f436-44a9-851d-b9e6f4c543bf-root\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.738070 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738076    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-textfile\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738104    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738122    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7dzx\" (UniqueName: \"kubernetes.io/projected/6149ebba-f436-44a9-851d-b9e6f4c543bf-kube-api-access-g7dzx\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738143    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738169    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnp7k\" (UniqueName: \"kubernetes.io/projected/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-api-access-lnp7k\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-tls\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738203    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/973f742f-b3da-44ee-b805-af2d1db8fdf0-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a983224-41c0-4cb5-974b-bc589225ca41-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738247    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738257    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/973f742f-b3da-44ee-b805-af2d1db8fdf0-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738263    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-wtmp\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738565    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6149ebba-f436-44a9-851d-b9e6f4c543bf-metrics-client-ca\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738684    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-wtmp\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.738696 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738685    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.739427 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738707    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.739427 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738733    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6149ebba-f436-44a9-851d-b9e6f4c543bf-root\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.739427 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738950    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-textfile\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.739427 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.738973    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6149ebba-f436-44a9-851d-b9e6f4c543bf-sys\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.739427 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.739083    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/973f742f-b3da-44ee-b805-af2d1db8fdf0-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.739427 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:48:11.739188    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 19:48:11.739427 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:48:11.739241    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-tls podName:6149ebba-f436-44a9-851d-b9e6f4c543bf nodeName:}" failed. No retries permitted until 2026-07-28 19:48:12.239222885 +0000 UTC m=+73.419739935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-tls") pod "node-exporter-lbz5l" (UID: "6149ebba-f436-44a9-851d-b9e6f4c543bf") : secret "node-exporter-tls" not found
Jul 28 19:48:11.739805 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.739633    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a983224-41c0-4cb5-974b-bc589225ca41-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.741065 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.741044    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.741176 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.741081    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a983224-41c0-4cb5-974b-bc589225ca41-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.741348 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.741323    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.741419 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.741382    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a983224-41c0-4cb5-974b-bc589225ca41-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.746862 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.746810    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx2sq\" (UniqueName: \"kubernetes.io/projected/4a983224-41c0-4cb5-974b-bc589225ca41-kube-api-access-qx2sq\") pod \"openshift-state-metrics-b8f49f5c6-5v5lr\" (UID: \"4a983224-41c0-4cb5-974b-bc589225ca41\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.746862 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.746812    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnp7k\" (UniqueName: \"kubernetes.io/projected/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-api-access-lnp7k\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:11.747007 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.746899    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7dzx\" (UniqueName: \"kubernetes.io/projected/6149ebba-f436-44a9-851d-b9e6f4c543bf-kube-api-access-g7dzx\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:11.786399 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.786366    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:11.819360 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.819290    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:11.819360 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.819352    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:11.824656 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.824631    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:11.839043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.839012    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"
Jul 28 19:48:11.970570 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:11.970538    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:12.071801 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.071727    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:12.242750 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.242710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-tls\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:12.242957 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.242782    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:12.242957 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:48:12.242863    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 19:48:12.242957 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:48:12.242932    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-tls podName:6149ebba-f436-44a9-851d-b9e6f4c543bf nodeName:}" failed. No retries permitted until 2026-07-28 19:48:13.242914794 +0000 UTC m=+74.423431828 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-tls") pod "node-exporter-lbz5l" (UID: "6149ebba-f436-44a9-851d-b9e6f4c543bf") : secret "node-exporter-tls" not found
Jul 28 19:48:12.245351 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.245327    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/973f742f-b3da-44ee-b805-af2d1db8fdf0-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-956jc\" (UID: \"973f742f-b3da-44ee-b805-af2d1db8fdf0\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:12.260933 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.260906    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:12.402361 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.402260    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:12.457467 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.457443    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"
Jul 28 19:48:12.520223 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.520188    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:12.599699 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.599654    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:12.601770 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.601735    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:48:12.605347 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.605324    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.607156 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.607128    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 19:48:12.607312 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.607194    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 19:48:12.607312 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.607260    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 19:48:12.607563 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.607509    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 19:48:12.607563 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.607531    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 19:48:12.607563 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.607535    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 19:48:12.607949 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.607801    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 19:48:12.607949 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.607854    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 19:48:12.607949 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.607900    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-q6skf\""
Jul 28 19:48:12.607949 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.607930    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 19:48:12.621104 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.621083    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:48:12.646891 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.646851    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/433ab26f-8e8f-438e-afdd-328326afce93-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647015 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.646900    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-web-config\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647015 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.646920    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk68h\" (UniqueName: \"kubernetes.io/projected/433ab26f-8e8f-438e-afdd-328326afce93-kube-api-access-tk68h\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647015 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.646949    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-config-volume\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647152 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.647037    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647152 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.647073    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/433ab26f-8e8f-438e-afdd-328326afce93-config-out\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647152 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.647104    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647152 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.647133    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/433ab26f-8e8f-438e-afdd-328326afce93-tls-assets\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647371 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.647163    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647371 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.647213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/433ab26f-8e8f-438e-afdd-328326afce93-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647371 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.647250    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433ab26f-8e8f-438e-afdd-328326afce93-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647371 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.647318    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.647553 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.647407    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.685428 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.685402    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:12.704362 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.704337    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:12.748257 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748227    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.748663 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748268    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/433ab26f-8e8f-438e-afdd-328326afce93-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.748663 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748313    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433ab26f-8e8f-438e-afdd-328326afce93-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.748663 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748352    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.748663 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748401    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.748663 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748426    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/433ab26f-8e8f-438e-afdd-328326afce93-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.748663 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748456    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-web-config\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.748663 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748480    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk68h\" (UniqueName: \"kubernetes.io/projected/433ab26f-8e8f-438e-afdd-328326afce93-kube-api-access-tk68h\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.748663 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748519    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-config-volume\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.748663 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748595    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.748663 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748668    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/433ab26f-8e8f-438e-afdd-328326afce93-config-out\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.750746 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748690    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.750746 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.748741    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/433ab26f-8e8f-438e-afdd-328326afce93-tls-assets\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.753345 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:48:12.752185    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/433ab26f-8e8f-438e-afdd-328326afce93-alertmanager-trusted-ca-bundle podName:433ab26f-8e8f-438e-afdd-328326afce93 nodeName:}" failed. No retries permitted until 2026-07-28 19:48:13.252160908 +0000 UTC m=+74.432677946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/433ab26f-8e8f-438e-afdd-328326afce93-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "433ab26f-8e8f-438e-afdd-328326afce93") : configmap references non-existent config key: ca-bundle.crt
Jul 28 19:48:12.753345 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.752975    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/433ab26f-8e8f-438e-afdd-328326afce93-tls-assets\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.753345 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.753083    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/433ab26f-8e8f-438e-afdd-328326afce93-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.754567 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.753646    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/433ab26f-8e8f-438e-afdd-328326afce93-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.754567 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.753828    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/433ab26f-8e8f-438e-afdd-328326afce93-config-out\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.755375 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.755001    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-config-volume\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.757182 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.757045    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.757182 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.757071    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.759574 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.759472    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-web-config\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.759927 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.759904    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.760709 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.760684    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.762159 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.762123    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/433ab26f-8e8f-438e-afdd-328326afce93-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.773826 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.773784    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk68h\" (UniqueName: \"kubernetes.io/projected/433ab26f-8e8f-438e-afdd-328326afce93-kube-api-access-tk68h\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:12.823482 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.822461    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:12.897740 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.897665    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-956jc"]
Jul 28 19:48:12.902159 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:12.902132    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod973f742f_b3da_44ee_b805_af2d1db8fdf0.slice/crio-989c6a73bab32e9a7624b7f4f2a55245c0bfdb55ef048c9a9abf9123fa2173b7 WatchSource:0}: Error finding container 989c6a73bab32e9a7624b7f4f2a55245c0bfdb55ef048c9a9abf9123fa2173b7: Status 404 returned error can't find the container with id 989c6a73bab32e9a7624b7f4f2a55245c0bfdb55ef048c9a9abf9123fa2173b7
Jul 28 19:48:12.908402 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:12.908233    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr"]
Jul 28 19:48:12.910387 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:12.910361    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a983224_41c0_4cb5_974b_bc589225ca41.slice/crio-f96f847641e2147b3dfa4de934b3492460993c3e80042e1814d00cb55598561a WatchSource:0}: Error finding container f96f847641e2147b3dfa4de934b3492460993c3e80042e1814d00cb55598561a: Status 404 returned error can't find the container with id f96f847641e2147b3dfa4de934b3492460993c3e80042e1814d00cb55598561a
Jul 28 19:48:13.254241 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.254167    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433ab26f-8e8f-438e-afdd-328326afce93-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:13.254241 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.254202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-tls\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:13.254916 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.254889    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/433ab26f-8e8f-438e-afdd-328326afce93-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"433ab26f-8e8f-438e-afdd-328326afce93\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:13.256390 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.256365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6149ebba-f436-44a9-851d-b9e6f4c543bf-node-exporter-tls\") pod \"node-exporter-lbz5l\" (UID: \"6149ebba-f436-44a9-851d-b9e6f4c543bf\") " pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:13.366295 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.366254    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lbz5l"
Jul 28 19:48:13.375989 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:13.375963    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6149ebba_f436_44a9_851d_b9e6f4c543bf.slice/crio-05ff5681088918cee1c43f0cb8c1578ab162d3c4d0ce60a203bb971002e76334 WatchSource:0}: Error finding container 05ff5681088918cee1c43f0cb8c1578ab162d3c4d0ce60a203bb971002e76334: Status 404 returned error can't find the container with id 05ff5681088918cee1c43f0cb8c1578ab162d3c4d0ce60a203bb971002e76334
Jul 28 19:48:13.517476 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.517423    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 19:48:13.648402 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.648369    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 19:48:13.654384 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:13.654359    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod433ab26f_8e8f_438e_afdd_328326afce93.slice/crio-cedddbc2485972773855e7a8c1b8c165e206d411dca373d215cc858c41d52db7 WatchSource:0}: Error finding container cedddbc2485972773855e7a8c1b8c165e206d411dca373d215cc858c41d52db7: Status 404 returned error can't find the container with id cedddbc2485972773855e7a8c1b8c165e206d411dca373d215cc858c41d52db7
Jul 28 19:48:13.684751 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.684717    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hcb5b" event={"ID":"4028a654-3327-4719-b4d0-6434b1573bbd","Type":"ContainerStarted","Data":"bcd7a5d99a9bb1569ac9056ad8c718ad49d6f03f61446f5a5d5a321241d11fee"}
Jul 28 19:48:13.686139 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.686114    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"433ab26f-8e8f-438e-afdd-328326afce93","Type":"ContainerStarted","Data":"cedddbc2485972773855e7a8c1b8c165e206d411dca373d215cc858c41d52db7"}
Jul 28 19:48:13.687422 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.687398    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lbz5l" event={"ID":"6149ebba-f436-44a9-851d-b9e6f4c543bf","Type":"ContainerStarted","Data":"05ff5681088918cee1c43f0cb8c1578ab162d3c4d0ce60a203bb971002e76334"}
Jul 28 19:48:13.688817 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.688781    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc" event={"ID":"973f742f-b3da-44ee-b805-af2d1db8fdf0","Type":"ContainerStarted","Data":"989c6a73bab32e9a7624b7f4f2a55245c0bfdb55ef048c9a9abf9123fa2173b7"}
Jul 28 19:48:13.690384 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.690359    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qcl5s" event={"ID":"8a92e62e-b631-4596-8efc-b4e65ff8e290","Type":"ContainerStarted","Data":"391a8945c90b16492f8cfe4b904147781c68da15c0e50c929d1d639a887b0bff"}
Jul 28 19:48:13.690512 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.690410    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:48:13.692533 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.692512    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr" event={"ID":"4a983224-41c0-4cb5-974b-bc589225ca41","Type":"ContainerStarted","Data":"19259aefeea1afd7c1e8d56431db45c2071d309f3d9ab19feb37e3a5bf360eaf"}
Jul 28 19:48:13.692637 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.692563    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr" event={"ID":"4a983224-41c0-4cb5-974b-bc589225ca41","Type":"ContainerStarted","Data":"c0e9f192439a4b959e7f026d65269368621b094022ef251664b8c9730762d62e"}
Jul 28 19:48:13.692637 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.692594    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr" event={"ID":"4a983224-41c0-4cb5-974b-bc589225ca41","Type":"ContainerStarted","Data":"f96f847641e2147b3dfa4de934b3492460993c3e80042e1814d00cb55598561a"}
Jul 28 19:48:13.698465 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.698419    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hcb5b" podStartSLOduration=70.407671394 podStartE2EDuration="1m14.698402561s" totalImagesPullingTime="4.290731167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:13.697165177 +0000 UTC m=+74.877682234" watchObservedRunningTime="2026-07-28 19:48:13.698402561 +0000 UTC m=+74.878919598"
Jul 28 19:48:13.713345 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:13.713310    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qcl5s" podStartSLOduration=68.526293606 podStartE2EDuration="1m14.71329423s" totalImagesPullingTime="6.187000624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:13.712133458 +0000 UTC m=+74.892650516" watchObservedRunningTime="2026-07-28 19:48:13.71329423 +0000 UTC m=+74.893811278"
Jul 28 19:48:14.554156 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.554076    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:14.559263 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.559236    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"]
Jul 28 19:48:14.564434 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.564250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.567272 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.567253    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 19:48:14.567394 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.567274    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 19:48:14.567394 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.567338    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 19:48:14.567499 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.567254    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 19:48:14.567748 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.567605    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 19:48:14.567748 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.567643    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2dr77eo3h6edu\""
Jul 28 19:48:14.567894 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.567808    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lnhtf\""
Jul 28 19:48:14.578391 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.578370    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"]
Jul 28 19:48:14.659942 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.659907    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:14.664083 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.664050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-grpc-tls\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.664207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.664099    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.664207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.664134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.664207 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.664198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-metrics-client-ca\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.664405 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.664227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-tls\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.664405 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.664347    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qw68\" (UniqueName: \"kubernetes.io/projected/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-kube-api-access-5qw68\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.664405 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.664392    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.664561 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.664423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.761156 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.761112    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:14.765697 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.765675    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qw68\" (UniqueName: \"kubernetes.io/projected/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-kube-api-access-5qw68\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.765808 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.765716    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.765892 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.765869    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.765963 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.765926    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-grpc-tls\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.766020 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.765973    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.766020 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.766021    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.766593 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.766200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-metrics-client-ca\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.766593 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.766251    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-tls\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.767074 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.767020    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-metrics-client-ca\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.768554 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.768531    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.768713 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.768689    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.768884 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.768863    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-grpc-tls\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.769022 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.769000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.769393 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.769376    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-tls\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.769462 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.769401    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.777245 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.777225    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qw68\" (UniqueName: \"kubernetes.io/projected/8b4515b9-2bf8-4567-8a34-d90f74e2de2c-kube-api-access-5qw68\") pod \"thanos-querier-5b77bc96c6-z2s7q\" (UID: \"8b4515b9-2bf8-4567-8a34-d90f74e2de2c\") " pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:14.863393 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.863309    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:14.876786 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:14.876759    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:15.177064 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.176971    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"]
Jul 28 19:48:15.180780 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:15.180740    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4515b9_2bf8_4567_8a34_d90f74e2de2c.slice/crio-3742227cbf0c89478bf9f0d77d0989e0d0bcacc35a291aa4db08bbb2f54291c4 WatchSource:0}: Error finding container 3742227cbf0c89478bf9f0d77d0989e0d0bcacc35a291aa4db08bbb2f54291c4: Status 404 returned error can't find the container with id 3742227cbf0c89478bf9f0d77d0989e0d0bcacc35a291aa4db08bbb2f54291c4
Jul 28 19:48:15.210236 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.210106    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:15.267236 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.267201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:15.409891 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.409805    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:15.468832 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.468796    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:15.702314 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.702262    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr" event={"ID":"4a983224-41c0-4cb5-974b-bc589225ca41","Type":"ContainerStarted","Data":"41d95f382ab1aa760ddb488bd8beba5f7ac21fb4c29a387d2d35883fde61568b"}
Jul 28 19:48:15.703876 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.703853    2569 generic.go:356] "Generic (PLEG): container finished" podID="433ab26f-8e8f-438e-afdd-328326afce93" containerID="95f0866a9e593abef03ed05c47b181f29e4f2f6c25bd24c78315200d189034e0" exitCode=0
Jul 28 19:48:15.703999 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.703901    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"433ab26f-8e8f-438e-afdd-328326afce93","Type":"ContainerDied","Data":"95f0866a9e593abef03ed05c47b181f29e4f2f6c25bd24c78315200d189034e0"}
Jul 28 19:48:15.708156 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.708127    2569 generic.go:356] "Generic (PLEG): container finished" podID="6149ebba-f436-44a9-851d-b9e6f4c543bf" containerID="897eff986eb2c9b07f3cf265953ba5a7fb4eb83fd010bc335c6e09e6cb0bbb05" exitCode=0
Jul 28 19:48:15.708264 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.708227    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lbz5l" event={"ID":"6149ebba-f436-44a9-851d-b9e6f4c543bf","Type":"ContainerDied","Data":"897eff986eb2c9b07f3cf265953ba5a7fb4eb83fd010bc335c6e09e6cb0bbb05"}
Jul 28 19:48:15.711064 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.711028    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc" event={"ID":"973f742f-b3da-44ee-b805-af2d1db8fdf0","Type":"ContainerStarted","Data":"a0704d57667be7b277510cd6fa60443680e4b969c018cfae89f815670647dbfb"}
Jul 28 19:48:15.711146 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.711080    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc" event={"ID":"973f742f-b3da-44ee-b805-af2d1db8fdf0","Type":"ContainerStarted","Data":"f9ac2c9c4f8efdb3380aa7ae842ea3623b571e96dfc45e94abc5c05d9640f52c"}
Jul 28 19:48:15.711146 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.711110    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc" event={"ID":"973f742f-b3da-44ee-b805-af2d1db8fdf0","Type":"ContainerStarted","Data":"036812bcd46a6fddd9f62508dd663bf8b008a24db43881504e3d7a58d17abacd"}
Jul 28 19:48:15.712133 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.712113    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q" event={"ID":"8b4515b9-2bf8-4567-8a34-d90f74e2de2c","Type":"ContainerStarted","Data":"3742227cbf0c89478bf9f0d77d0989e0d0bcacc35a291aa4db08bbb2f54291c4"}
Jul 28 19:48:15.718535 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.718476    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5v5lr" podStartSLOduration=2.727861017 podStartE2EDuration="4.718462262s" totalImagesPullingTime="1.990601245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:15.717725324 +0000 UTC m=+76.898242379" watchObservedRunningTime="2026-07-28 19:48:15.718462262 +0000 UTC m=+76.898979319"
Jul 28 19:48:15.750564 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.750526    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-956jc" podStartSLOduration=2.630202299 podStartE2EDuration="4.750511559s" totalImagesPullingTime="2.12030926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:15.749087571 +0000 UTC m=+76.929604626" watchObservedRunningTime="2026-07-28 19:48:15.750511559 +0000 UTC m=+76.931028617"
Jul 28 19:48:15.929661 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.929628    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5b7c949b84-q66pt"]
Jul 28 19:48:15.932721 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.932696    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:15.934851 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.934827    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 19:48:15.934968 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.934892    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-rs6nw\""
Jul 28 19:48:15.934968 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.934896    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 19:48:15.935231 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.935216    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 19:48:15.935309 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.935251    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-etihpehopigd1\""
Jul 28 19:48:15.935309 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.935254    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 19:48:15.942208 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.942189    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b7c949b84-q66pt"]
Jul 28 19:48:15.977229 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.977167    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/33e28913-f966-4ddb-93ac-d9887d8ea767-secret-metrics-server-client-certs\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:15.977229 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.977216    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/33e28913-f966-4ddb-93ac-d9887d8ea767-metrics-server-audit-profiles\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:15.977415 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.977240    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/33e28913-f966-4ddb-93ac-d9887d8ea767-secret-metrics-server-tls\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:15.977415 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.977265    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/33e28913-f966-4ddb-93ac-d9887d8ea767-audit-log\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:15.977415 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.977320    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mm7h\" (UniqueName: \"kubernetes.io/projected/33e28913-f966-4ddb-93ac-d9887d8ea767-kube-api-access-7mm7h\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:15.977543 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.977447    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33e28913-f966-4ddb-93ac-d9887d8ea767-client-ca-bundle\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:15.977581 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:15.977537    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33e28913-f966-4ddb-93ac-d9887d8ea767-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.078442 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.078416    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/33e28913-f966-4ddb-93ac-d9887d8ea767-audit-log\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.078561 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.078454    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mm7h\" (UniqueName: \"kubernetes.io/projected/33e28913-f966-4ddb-93ac-d9887d8ea767-kube-api-access-7mm7h\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.078561 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.078482    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33e28913-f966-4ddb-93ac-d9887d8ea767-client-ca-bundle\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.078561 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.078521    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33e28913-f966-4ddb-93ac-d9887d8ea767-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.078561 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.078543    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/33e28913-f966-4ddb-93ac-d9887d8ea767-secret-metrics-server-client-certs\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.078760 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.078576    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/33e28913-f966-4ddb-93ac-d9887d8ea767-metrics-server-audit-profiles\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.078760 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.078705    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/33e28913-f966-4ddb-93ac-d9887d8ea767-secret-metrics-server-tls\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.078955 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.078924    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/33e28913-f966-4ddb-93ac-d9887d8ea767-audit-log\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.079318 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.079272    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33e28913-f966-4ddb-93ac-d9887d8ea767-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.079567 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.079542    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/33e28913-f966-4ddb-93ac-d9887d8ea767-metrics-server-audit-profiles\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.081089 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.081069    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/33e28913-f966-4ddb-93ac-d9887d8ea767-secret-metrics-server-tls\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.081588 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.081564    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/33e28913-f966-4ddb-93ac-d9887d8ea767-secret-metrics-server-client-certs\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.081677 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.081629    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33e28913-f966-4ddb-93ac-d9887d8ea767-client-ca-bundle\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.085172 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.085150    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mm7h\" (UniqueName: \"kubernetes.io/projected/33e28913-f966-4ddb-93ac-d9887d8ea767-kube-api-access-7mm7h\") pod \"metrics-server-5b7c949b84-q66pt\" (UID: \"33e28913-f966-4ddb-93ac-d9887d8ea767\") " pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.242178 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.242112    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:16.319321 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.319273    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"]
Jul 28 19:48:16.320146 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.320073    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:16.324528 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.324405    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"
Jul 28 19:48:16.327131 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.327110    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cwvj8\""
Jul 28 19:48:16.327438 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.327420    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 19:48:16.330892 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.330837    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"]
Jul 28 19:48:16.381057 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.381026    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fa37ab8f-afd8-4062-bf21-5cc435623849-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rm5t5\" (UID: \"fa37ab8f-afd8-4062-bf21-5cc435623849\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"
Jul 28 19:48:16.381210 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.381086    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x24xc\" (UniqueName: \"kubernetes.io/projected/fa37ab8f-afd8-4062-bf21-5cc435623849-kube-api-access-x24xc\") pod \"monitoring-plugin-68b7fc9574-rm5t5\" (UID: \"fa37ab8f-afd8-4062-bf21-5cc435623849\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"
Jul 28 19:48:16.401479 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.401409    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b7c949b84-q66pt"]
Jul 28 19:48:16.403509 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.403463    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:16.405259 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:16.405235    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e28913_f966_4ddb_93ac_d9887d8ea767.slice/crio-b8749dadb6a5d2d854eb5ba17d3045033e4536b7a2e02662b7bbfda3ae5378a7 WatchSource:0}: Error finding container b8749dadb6a5d2d854eb5ba17d3045033e4536b7a2e02662b7bbfda3ae5378a7: Status 404 returned error can't find the container with id b8749dadb6a5d2d854eb5ba17d3045033e4536b7a2e02662b7bbfda3ae5378a7
Jul 28 19:48:16.459849 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.459827    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:16.482595 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.482403    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fa37ab8f-afd8-4062-bf21-5cc435623849-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rm5t5\" (UID: \"fa37ab8f-afd8-4062-bf21-5cc435623849\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"
Jul 28 19:48:16.482595 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.482454    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x24xc\" (UniqueName: \"kubernetes.io/projected/fa37ab8f-afd8-4062-bf21-5cc435623849-kube-api-access-x24xc\") pod \"monitoring-plugin-68b7fc9574-rm5t5\" (UID: \"fa37ab8f-afd8-4062-bf21-5cc435623849\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"
Jul 28 19:48:16.485455 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.485433    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fa37ab8f-afd8-4062-bf21-5cc435623849-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rm5t5\" (UID: \"fa37ab8f-afd8-4062-bf21-5cc435623849\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"
Jul 28 19:48:16.491592 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.491538    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x24xc\" (UniqueName: \"kubernetes.io/projected/fa37ab8f-afd8-4062-bf21-5cc435623849-kube-api-access-x24xc\") pod \"monitoring-plugin-68b7fc9574-rm5t5\" (UID: \"fa37ab8f-afd8-4062-bf21-5cc435623849\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"
Jul 28 19:48:16.638608 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.638527    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:16.646720 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.646697    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"
Jul 28 19:48:16.722893 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.722868    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-769445799f-6j6h5"]
Jul 28 19:48:16.730407 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.729118    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lbz5l" event={"ID":"6149ebba-f436-44a9-851d-b9e6f4c543bf","Type":"ContainerStarted","Data":"7e2b55044fb5bb75280e706fa2631772d41e9c0b2f5c7e0686dac7455b1d957b"}
Jul 28 19:48:16.730407 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.729172    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lbz5l" event={"ID":"6149ebba-f436-44a9-851d-b9e6f4c543bf","Type":"ContainerStarted","Data":"72b043e3bf303b05a2775b2b06cffe10442f63ecadf944d933658b365bc52346"}
Jul 28 19:48:16.730407 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.729209    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt" event={"ID":"33e28913-f966-4ddb-93ac-d9887d8ea767","Type":"ContainerStarted","Data":"b8749dadb6a5d2d854eb5ba17d3045033e4536b7a2e02662b7bbfda3ae5378a7"}
Jul 28 19:48:16.730407 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.729356    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.731700 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.731673    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 19:48:16.731911 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.731888    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 19:48:16.733591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.732123    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 19:48:16.733591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.732189    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 19:48:16.733591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.732789    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 19:48:16.733898 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.732492    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-n94kd\""
Jul 28 19:48:16.739647 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.738927    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 19:48:16.742673 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.742531    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-769445799f-6j6h5"]
Jul 28 19:48:16.742817 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.742773    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lbz5l" podStartSLOduration=4.014707146 podStartE2EDuration="5.742755194s" totalImagesPullingTime="1.643613796s" totalInitContainerRuntime="84.434252ms" isStatefulPod=true podCreationTimestamp="2026-07-28 19:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:16.741966243 +0000 UTC m=+77.922483300" watchObservedRunningTime="2026-07-28 19:48:16.742755194 +0000 UTC m=+77.923272251"
Jul 28 19:48:16.762389 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.762363    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:16.786001 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.785480    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-secret-telemeter-client\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.786001 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.785671    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.786001 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.785698    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.786001 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.785781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-metrics-client-ca\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.786001 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.785820    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-telemeter-client-tls\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.786001 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.785842    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptk5m\" (UniqueName: \"kubernetes.io/projected/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-kube-api-access-ptk5m\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.786469 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.786199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-federate-client-tls\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.786469 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.786264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-serving-certs-ca-bundle\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.803000 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.802952    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"]
Jul 28 19:48:16.886764 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.886726    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-metrics-client-ca\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.886940 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.886784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-telemeter-client-tls\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.886940 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.886815    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptk5m\" (UniqueName: \"kubernetes.io/projected/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-kube-api-access-ptk5m\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.886940 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.886870    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-federate-client-tls\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.886940 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.886898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-serving-certs-ca-bundle\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.886940 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.886936    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-secret-telemeter-client\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.887220 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.887143    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.887220 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.887179    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.887614 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.887588    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-metrics-client-ca\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.887720 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.887673    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-serving-certs-ca-bundle\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.888699 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.888617    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.890150 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.889820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-telemeter-client-tls\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.890577 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.890531    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-federate-client-tls\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.890577 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.890561    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-secret-telemeter-client\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.891154 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.891115    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:16.894771 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:16.894746    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptk5m\" (UniqueName: \"kubernetes.io/projected/8841f1be-3558-4ac1-bb6d-bf3e38f2b51f-kube-api-access-ptk5m\") pod \"telemeter-client-769445799f-6j6h5\" (UID: \"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f\") " pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:17.042932 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.042897    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-769445799f-6j6h5"
Jul 28 19:48:17.144943 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:17.144864    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa37ab8f_afd8_4062_bf21_5cc435623849.slice/crio-f60a40053d0387c21bdf0190ceb3b00ae3ff5121677762023c816d1c46afc2d7 WatchSource:0}: Error finding container f60a40053d0387c21bdf0190ceb3b00ae3ff5121677762023c816d1c46afc2d7: Status 404 returned error can't find the container with id f60a40053d0387c21bdf0190ceb3b00ae3ff5121677762023c816d1c46afc2d7
Jul 28 19:48:17.292586 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.292521    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-769445799f-6j6h5"]
Jul 28 19:48:17.735938 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.735862    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5" event={"ID":"fa37ab8f-afd8-4062-bf21-5cc435623849","Type":"ContainerStarted","Data":"f60a40053d0387c21bdf0190ceb3b00ae3ff5121677762023c816d1c46afc2d7"}
Jul 28 19:48:17.820923 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:48:17.820881    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8841f1be_3558_4ac1_bb6d_bf3e38f2b51f.slice/crio-3ef6959df61904ff5090b9db478f010d19ce4412dea6e5a4b05c71589af74a62 WatchSource:0}: Error finding container 3ef6959df61904ff5090b9db478f010d19ce4412dea6e5a4b05c71589af74a62: Status 404 returned error can't find the container with id 3ef6959df61904ff5090b9db478f010d19ce4412dea6e5a4b05c71589af74a62
Jul 28 19:48:17.849611 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.849563    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:17.851152 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.850511    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:48:17.855741 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.855719    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:17.857931 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.857908    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 19:48:17.858362 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.858305    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5771iffc9nre5\""
Jul 28 19:48:17.858477 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.858369    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 19:48:17.858477 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.858374    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 19:48:17.858595 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.858538    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 19:48:17.858595 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.858543    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 19:48:17.859753 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.858867    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 19:48:17.859753 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.858973    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 19:48:17.859753 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.859176    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pcbk9\""
Jul 28 19:48:17.859753 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.859195    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 19:48:17.859753 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.859327    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 19:48:17.859753 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.859339    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 19:48:17.859753 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.859519    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 19:48:17.861142 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.861120    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 19:48:17.874888 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.874866    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 19:48:17.878060 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.877945    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:48:17.879218 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.879196    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 19:48:17.999791 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.999710    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:17.999791 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.999769    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0d43a27-65c9-4966-83c2-dd85dd344c88-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.999817    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0d43a27-65c9-4966-83c2-dd85dd344c88-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.999844    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.999874    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.999891    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.999909    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-config\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.999949    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.999973    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000010 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:17.999995    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000429 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.000052    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000429 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.000084    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000429 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.000114    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000429 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.000152    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nq9d\" (UniqueName: \"kubernetes.io/projected/d0d43a27-65c9-4966-83c2-dd85dd344c88-kube-api-access-9nq9d\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000429 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.000193    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000429 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.000227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000429 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.000247    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000429 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.000274    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000429 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.000326    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.000429 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.000397    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.019407 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.019375    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.101753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-config\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.101815    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.101846    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.101871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.101915    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.101943    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.101973    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nq9d\" (UniqueName: \"kubernetes.io/projected/d0d43a27-65c9-4966-83c2-dd85dd344c88-kube-api-access-9nq9d\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102038    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102075    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102107    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102134    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102157    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102188    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102229    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102306    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0d43a27-65c9-4966-83c2-dd85dd344c88-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.103498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102341    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0d43a27-65c9-4966-83c2-dd85dd344c88-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.104516 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102368    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.104516 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102394    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.104516 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.102430    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.104516 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.103182    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.104516 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.103669    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.104516 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.103946    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.107471 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.107444    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.107997 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.107971    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.109398 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.108963    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.110564 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.110540    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.110738 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.110718    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0d43a27-65c9-4966-83c2-dd85dd344c88-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.113109 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.113024    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:18.115462 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.115439    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0d43a27-65c9-4966-83c2-dd85dd344c88-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.116003 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.115955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.118427 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.118360    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0d43a27-65c9-4966-83c2-dd85dd344c88-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.118979 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.118938    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nq9d\" (UniqueName: \"kubernetes.io/projected/d0d43a27-65c9-4966-83c2-dd85dd344c88-kube-api-access-9nq9d\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.119121 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.119095    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.119651 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.119612    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-config\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.120179 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.120157    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.120179 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.120169    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.120337 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.120266    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.120738 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.120714    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.120937 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.120913    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.123183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.123162    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0d43a27-65c9-4966-83c2-dd85dd344c88-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0d43a27-65c9-4966-83c2-dd85dd344c88\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.175144 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.175055    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:18.577817 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.577080    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:48:18.744982 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.744930    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"433ab26f-8e8f-438e-afdd-328326afce93","Type":"ContainerStarted","Data":"ef6eab1b83d9029d3fab316a86ecdb4b40ac50b4bcceabb24619d0c9b9de54b4"}
Jul 28 19:48:18.745364 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.745014    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"433ab26f-8e8f-438e-afdd-328326afce93","Type":"ContainerStarted","Data":"4b3f03d1bf756a0e8701eddd68bccf7aaf7a8bcd88c70bca88546abb8beac172"}
Jul 28 19:48:18.745364 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.745056    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"433ab26f-8e8f-438e-afdd-328326afce93","Type":"ContainerStarted","Data":"8dd771787999fa6f954dec855c60f6d0d95afb9e2ce8cf519ab5ccfde339b499"}
Jul 28 19:48:18.748645 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.748478    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q" event={"ID":"8b4515b9-2bf8-4567-8a34-d90f74e2de2c","Type":"ContainerStarted","Data":"9c7d12af37cfcc4b0ca0767dfce511d4115a7a517c94d6e53085cb85954a46c3"}
Jul 28 19:48:18.748645 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.748551    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q" event={"ID":"8b4515b9-2bf8-4567-8a34-d90f74e2de2c","Type":"ContainerStarted","Data":"c44a23451a293e7c7bb0ca0b3117b2f24a3cf112c5fea083d48fdc1589026bcc"}
Jul 28 19:48:18.748645 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.748590    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q" event={"ID":"8b4515b9-2bf8-4567-8a34-d90f74e2de2c","Type":"ContainerStarted","Data":"028abbeb24cfb28d761f275d024cbfce787105c8ce0fc21e40bfef3563a5e2ad"}
Jul 28 19:48:18.750901 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.750879    2569 generic.go:356] "Generic (PLEG): container finished" podID="d0d43a27-65c9-4966-83c2-dd85dd344c88" containerID="a7922af7712809cefffde3f7c99273e35ccf8445befa153399aef0085c7b46d3" exitCode=0
Jul 28 19:48:18.751000 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.750973    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0d43a27-65c9-4966-83c2-dd85dd344c88","Type":"ContainerDied","Data":"a7922af7712809cefffde3f7c99273e35ccf8445befa153399aef0085c7b46d3"}
Jul 28 19:48:18.751072 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.751031    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0d43a27-65c9-4966-83c2-dd85dd344c88","Type":"ContainerStarted","Data":"7fd9e5c282218f705365ebb4a0872892e0c6d80d5f83e20b63815ba7b0379c61"}
Jul 28 19:48:18.752972 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.752935    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-769445799f-6j6h5" event={"ID":"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f","Type":"ContainerStarted","Data":"3ef6959df61904ff5090b9db478f010d19ce4412dea6e5a4b05c71589af74a62"}
Jul 28 19:48:18.757098 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.756579    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt" event={"ID":"33e28913-f966-4ddb-93ac-d9887d8ea767","Type":"ContainerStarted","Data":"e25ffa6fc69e4af982d6f5f31f6d6d59abb691bd55755540c0d915673eeeec99"}
Jul 28 19:48:18.790560 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:18.790511    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt" podStartSLOduration=1.7350699280000002 podStartE2EDuration="3.790493753s" totalImagesPullingTime="2.055423825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:18.788727954 +0000 UTC m=+79.969245011" watchObservedRunningTime="2026-07-28 19:48:18.790493753 +0000 UTC m=+79.971010788"
Jul 28 19:48:19.765183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:19.765143    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"433ab26f-8e8f-438e-afdd-328326afce93","Type":"ContainerStarted","Data":"54b9243db085446b031a8f966057cf882903e07bcc10c437a4dba8d9f5e59e8a"}
Jul 28 19:48:19.765610 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:19.765230    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"433ab26f-8e8f-438e-afdd-328326afce93","Type":"ContainerStarted","Data":"ca8e37127dda3d5f2560f96e8b1723cc9d40a8479366f1b2992097e111cabef0"}
Jul 28 19:48:19.766790 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:19.766761    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5" event={"ID":"fa37ab8f-afd8-4062-bf21-5cc435623849","Type":"ContainerStarted","Data":"7a94ae2157c66a1bef4388d62566afeb6b7eeeefd00555901b5e531d41cad942"}
Jul 28 19:48:19.781672 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:19.781626    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5" podStartSLOduration=1.731703692 podStartE2EDuration="3.781612106s" totalImagesPullingTime="2.049908414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:19.779771904 +0000 UTC m=+80.960288961" watchObservedRunningTime="2026-07-28 19:48:19.781612106 +0000 UTC m=+80.962129140"
Jul 28 19:48:20.774614 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.774577    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"433ab26f-8e8f-438e-afdd-328326afce93","Type":"ContainerStarted","Data":"9b01b1ead667a0a6e4cfdd2323fa6cf8519bd361c9088d0c499beb06b56dc2f7"}
Jul 28 19:48:20.777815 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.777783    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q" event={"ID":"8b4515b9-2bf8-4567-8a34-d90f74e2de2c","Type":"ContainerStarted","Data":"e25b09c52e697890618813d38c2cafcbea283e4c6ff8f3d2471422302d4379b4"}
Jul 28 19:48:20.777952 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.777865    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:20.777952 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.777898    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q" event={"ID":"8b4515b9-2bf8-4567-8a34-d90f74e2de2c","Type":"ContainerStarted","Data":"7198925e711d1c31466cb48d1ff31da88dc63564fa4c34d08638b49ba521a3d5"}
Jul 28 19:48:20.777952 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.777936    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q" event={"ID":"8b4515b9-2bf8-4567-8a34-d90f74e2de2c","Type":"ContainerStarted","Data":"2656ab5fe06319b6ed7bbb6f288f578fdcfbfb9ec78feefaf23aca35610051c6"}
Jul 28 19:48:20.780786 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.780761    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-769445799f-6j6h5" event={"ID":"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f","Type":"ContainerStarted","Data":"4f71f9a6f221254f98fcde4738a2098a2daae3e301009ad47aebc1b51df038e6"}
Jul 28 19:48:20.780894 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.780811    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-769445799f-6j6h5" event={"ID":"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f","Type":"ContainerStarted","Data":"e73af557e90f9793d77ad7a860831ed4e444562f8bec0c378f87710c7888ac15"}
Jul 28 19:48:20.780894 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.780835    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-769445799f-6j6h5" event={"ID":"8841f1be-3558-4ac1-bb6d-bf3e38f2b51f","Type":"ContainerStarted","Data":"6cc9cc388b3b0cc0eb8871fd99e71a617a9db59d73485346250b4c8bb290118e"}
Jul 28 19:48:20.780999 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.780929    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"
Jul 28 19:48:20.785397 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.785379    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rm5t5"
Jul 28 19:48:20.798334 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.798237    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.832977044 podStartE2EDuration="8.798222578s" totalImagesPullingTime="4.935094043s" totalInitContainerRuntime="30.151491ms" isStatefulPod=false podCreationTimestamp="2026-07-28 19:48:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:20.797111983 +0000 UTC m=+81.977629040" watchObservedRunningTime="2026-07-28 19:48:20.798222578 +0000 UTC m=+81.978739636"
Jul 28 19:48:20.816200 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.816156    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-769445799f-6j6h5" podStartSLOduration=2.57260402 podStartE2EDuration="4.81613789s" totalImagesPullingTime="2.24353387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:20.814912234 +0000 UTC m=+81.995429304" watchObservedRunningTime="2026-07-28 19:48:20.81613789 +0000 UTC m=+81.996654945"
Jul 28 19:48:20.853979 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:20.853934    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q" podStartSLOduration=2.203760726 podStartE2EDuration="6.853916585s" totalImagesPullingTime="4.650155859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:48:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:20.852652292 +0000 UTC m=+82.033169349" watchObservedRunningTime="2026-07-28 19:48:20.853916585 +0000 UTC m=+82.034433641"
Jul 28 19:48:21.670930 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:21.670807    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:21.728080 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:21.728049    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:21.837269 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:21.837239    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:22.369662 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:22.369624    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:22.461842 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:22.461807    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:22.797752 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:22.797716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0d43a27-65c9-4966-83c2-dd85dd344c88","Type":"ContainerStarted","Data":"af302d573bdab2d277c5d9a8ef61ae536ff52ed59486d9c68403acb11694f7d0"}
Jul 28 19:48:22.797882 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:22.797834    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0d43a27-65c9-4966-83c2-dd85dd344c88","Type":"ContainerStarted","Data":"7cdf4ae55c6cfa6dd2bc6acf169e9e22e744afefa99a6c27471a67956a617f97"}
Jul 28 19:48:22.797959 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:22.797880    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0d43a27-65c9-4966-83c2-dd85dd344c88","Type":"ContainerStarted","Data":"3357c56055bc5e6b8486aabc58bdcfcfefe08739d12603adda748ab5edc70bd3"}
Jul 28 19:48:23.105544 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:23.105496    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:23.206361 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:23.206333    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:23.805966 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:23.805934    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0d43a27-65c9-4966-83c2-dd85dd344c88","Type":"ContainerStarted","Data":"27f1813f0af28146e7c16eb4f5c635520261aa974bb2c8b63399bf5526f04064"}
Jul 28 19:48:23.806095 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:23.805997    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0d43a27-65c9-4966-83c2-dd85dd344c88","Type":"ContainerStarted","Data":"e779ac47fb357679d8cb8b8e76541dd1d33d1692729bfd85205149f987c4bb1b"}
Jul 28 19:48:23.806095 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:23.806022    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0d43a27-65c9-4966-83c2-dd85dd344c88","Type":"ContainerStarted","Data":"a16df3008131a8976d19aeaa471163c5f1c88a33b29082731b23b3f9cade9aed"}
Jul 28 19:48:23.831065 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:23.831016    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.014721952 podStartE2EDuration="6.831002268s" totalImagesPullingTime="3.785616283s" totalInitContainerRuntime="30.664033ms" isStatefulPod=false podCreationTimestamp="2026-07-28 19:48:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:48:23.830351345 +0000 UTC m=+85.010868406" watchObservedRunningTime="2026-07-28 19:48:23.831002268 +0000 UTC m=+85.011519324"
Jul 28 19:48:26.793049 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:26.793013    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b77bc96c6-z2s7q"
Jul 28 19:48:28.176008 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:28.175972    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:48:30.782981 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:30.782949    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:30.833133 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:30.833104    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:32.385259 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:32.385229    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66cb8cc574-4vcwl"]
Jul 28 19:48:33.097118 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:33.097085    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:33.151161 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:33.151126    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:36.243259 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:36.243221    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:36.243641 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:36.243294    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:41.626162 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:41.626130    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:41.690101 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:41.690073    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:44.700235 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:44.700205    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qcl5s"
Jul 28 19:48:45.776263 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:45.776229    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:45.828749 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:45.828722    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:53.039996 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:53.039966    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:53.140872 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:53.140843    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:53.189739 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:53.189717    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:56.249321 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:56.249287    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:56.253315 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:56.253292    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5b7c949b84-q66pt"
Jul 28 19:48:57.405651 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.405601    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66cb8cc574-4vcwl" podUID="503ff851-95fa-47c8-85e4-0fccec1dcba1" containerName="console" containerID="cri-o://e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1" gracePeriod=15
Jul 28 19:48:57.638424 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.638399    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:57.725397 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.725368    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-config\") pod \"503ff851-95fa-47c8-85e4-0fccec1dcba1\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") "
Jul 28 19:48:57.725536 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.725421    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-trusted-ca-bundle\") pod \"503ff851-95fa-47c8-85e4-0fccec1dcba1\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") "
Jul 28 19:48:57.725536 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.725458    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-service-ca\") pod \"503ff851-95fa-47c8-85e4-0fccec1dcba1\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") "
Jul 28 19:48:57.725536 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.725510    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-oauth-serving-cert\") pod \"503ff851-95fa-47c8-85e4-0fccec1dcba1\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") "
Jul 28 19:48:57.725708 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.725539    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-serving-cert\") pod \"503ff851-95fa-47c8-85e4-0fccec1dcba1\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") "
Jul 28 19:48:57.725708 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.725587    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm94h\" (UniqueName: \"kubernetes.io/projected/503ff851-95fa-47c8-85e4-0fccec1dcba1-kube-api-access-sm94h\") pod \"503ff851-95fa-47c8-85e4-0fccec1dcba1\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") "
Jul 28 19:48:57.725708 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.725635    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-oauth-config\") pod \"503ff851-95fa-47c8-85e4-0fccec1dcba1\" (UID: \"503ff851-95fa-47c8-85e4-0fccec1dcba1\") "
Jul 28 19:48:57.725882 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.725846    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-config" (OuterVolumeSpecName: "console-config") pod "503ff851-95fa-47c8-85e4-0fccec1dcba1" (UID: "503ff851-95fa-47c8-85e4-0fccec1dcba1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:48:57.725946 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.725915    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-service-ca" (OuterVolumeSpecName: "service-ca") pod "503ff851-95fa-47c8-85e4-0fccec1dcba1" (UID: "503ff851-95fa-47c8-85e4-0fccec1dcba1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:48:57.725994 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.725931    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "503ff851-95fa-47c8-85e4-0fccec1dcba1" (UID: "503ff851-95fa-47c8-85e4-0fccec1dcba1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:48:57.725994 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.725944    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "503ff851-95fa-47c8-85e4-0fccec1dcba1" (UID: "503ff851-95fa-47c8-85e4-0fccec1dcba1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:48:57.727874 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.727845    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "503ff851-95fa-47c8-85e4-0fccec1dcba1" (UID: "503ff851-95fa-47c8-85e4-0fccec1dcba1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:48:57.727874 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.727857    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/503ff851-95fa-47c8-85e4-0fccec1dcba1-kube-api-access-sm94h" (OuterVolumeSpecName: "kube-api-access-sm94h") pod "503ff851-95fa-47c8-85e4-0fccec1dcba1" (UID: "503ff851-95fa-47c8-85e4-0fccec1dcba1"). InnerVolumeSpecName "kube-api-access-sm94h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:48:57.728009 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.727909    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "503ff851-95fa-47c8-85e4-0fccec1dcba1" (UID: "503ff851-95fa-47c8-85e4-0fccec1dcba1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:48:57.826410 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.826386    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 19:48:57.826410 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.826406    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-oauth-serving-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 19:48:57.826536 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.826414    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-serving-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 19:48:57.826536 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.826424    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm94h\" (UniqueName: \"kubernetes.io/projected/503ff851-95fa-47c8-85e4-0fccec1dcba1-kube-api-access-sm94h\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 19:48:57.826536 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.826434    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-oauth-config\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 19:48:57.826536 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.826442    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-console-config\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 19:48:57.826536 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.826450    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503ff851-95fa-47c8-85e4-0fccec1dcba1-trusted-ca-bundle\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 19:48:57.946084 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.946057    2569 generic.go:356] "Generic (PLEG): container finished" podID="503ff851-95fa-47c8-85e4-0fccec1dcba1" containerID="e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1" exitCode=0
Jul 28 19:48:57.946187 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.946096    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66cb8cc574-4vcwl"
Jul 28 19:48:57.946187 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.946136    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66cb8cc574-4vcwl" event={"ID":"503ff851-95fa-47c8-85e4-0fccec1dcba1","Type":"ContainerDied","Data":"e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1"}
Jul 28 19:48:57.946259 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.946190    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66cb8cc574-4vcwl" event={"ID":"503ff851-95fa-47c8-85e4-0fccec1dcba1","Type":"ContainerDied","Data":"4b3d8a96cb9d2d4c820cf0800bee4203085fd93749ce086ddef4aebadcecfb72"}
Jul 28 19:48:57.946259 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.946213    2569 scope.go:117] "RemoveContainer" containerID="e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1"
Jul 28 19:48:57.954981 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.954966    2569 scope.go:117] "RemoveContainer" containerID="e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1"
Jul 28 19:48:57.955214 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:48:57.955194    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1\": container with ID starting with e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1 not found: ID does not exist" containerID="e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1"
Jul 28 19:48:57.955259 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.955220    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1"} err="failed to get container status \"e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1\": rpc error: code = NotFound desc = could not find container \"e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1\": container with ID starting with e784d479112caeb9ed6dcc9911258f46cfd6297bc40413afcb729ad32c618ae1 not found: ID does not exist"
Jul 28 19:48:57.966095 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.966075    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66cb8cc574-4vcwl"]
Jul 28 19:48:57.969320 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:57.969302    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66cb8cc574-4vcwl"]
Jul 28 19:48:59.364602 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:48:59.364573    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="503ff851-95fa-47c8-85e4-0fccec1dcba1" path="/var/lib/kubelet/pods/503ff851-95fa-47c8-85e4-0fccec1dcba1/volumes"
Jul 28 19:49:18.175428 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:18.175393    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:49:18.194853 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:18.194824    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:49:19.049412 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:19.049383    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:49:20.516672 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:20.516636    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:27.458003 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:27.457969    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:27.698219 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:27.698186    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:27.799088 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:27.799019    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:39.619474 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:39.619418    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:39.666866 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:39.666838    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:42.458431 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:42.458390    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:42.520982 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:42.520956    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:45.785835 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:45.785799    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:45.889683 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:45.889652    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:45.987582 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:45.987553    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:53.810404 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:53.810332    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:53.911318 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:49:53.911271    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:21.634031 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:21.633992    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:21.728882 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:21.728851    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:21.860567 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:21.860538    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:28.501221 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:28.501166    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:28.601940 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:28.601909    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:48.224443 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:48.224389    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:49.006452 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:49.006422    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:51.828025 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:51.827993    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:51.929802 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:51.929770    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:54.060786 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.060756    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:54.161097 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.161066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:54.247415 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.247388    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9l2jj"]
Jul 28 19:50:54.247740 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.247726    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503ff851-95fa-47c8-85e4-0fccec1dcba1" containerName="console"
Jul 28 19:50:54.247793 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.247741    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="503ff851-95fa-47c8-85e4-0fccec1dcba1" containerName="console"
Jul 28 19:50:54.247834 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.247809    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="503ff851-95fa-47c8-85e4-0fccec1dcba1" containerName="console"
Jul 28 19:50:54.249693 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.249671    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj"
Jul 28 19:50:54.251915 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.251888    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-twl9x\""
Jul 28 19:50:54.252009 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.251925    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 19:50:54.252446 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.252424    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 19:50:54.258384 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.258360    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9l2jj"]
Jul 28 19:50:54.413079 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.413019    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee999ad3-bc3d-4520-83b9-7c1a8f33210b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9l2jj\" (UID: \"ee999ad3-bc3d-4520-83b9-7c1a8f33210b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj"
Jul 28 19:50:54.413079 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.413051    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99t86\" (UniqueName: \"kubernetes.io/projected/ee999ad3-bc3d-4520-83b9-7c1a8f33210b-kube-api-access-99t86\") pod \"cert-manager-webhook-6486ffd997-9l2jj\" (UID: \"ee999ad3-bc3d-4520-83b9-7c1a8f33210b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj"
Jul 28 19:50:54.514188 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.514162    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee999ad3-bc3d-4520-83b9-7c1a8f33210b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9l2jj\" (UID: \"ee999ad3-bc3d-4520-83b9-7c1a8f33210b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj"
Jul 28 19:50:54.514301 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.514196    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99t86\" (UniqueName: \"kubernetes.io/projected/ee999ad3-bc3d-4520-83b9-7c1a8f33210b-kube-api-access-99t86\") pod \"cert-manager-webhook-6486ffd997-9l2jj\" (UID: \"ee999ad3-bc3d-4520-83b9-7c1a8f33210b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj"
Jul 28 19:50:54.531078 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.531051    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee999ad3-bc3d-4520-83b9-7c1a8f33210b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9l2jj\" (UID: \"ee999ad3-bc3d-4520-83b9-7c1a8f33210b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj"
Jul 28 19:50:54.531155 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.531117    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99t86\" (UniqueName: \"kubernetes.io/projected/ee999ad3-bc3d-4520-83b9-7c1a8f33210b-kube-api-access-99t86\") pod \"cert-manager-webhook-6486ffd997-9l2jj\" (UID: \"ee999ad3-bc3d-4520-83b9-7c1a8f33210b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj"
Jul 28 19:50:54.570544 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.570522    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj"
Jul 28 19:50:54.612591 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.612565    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:54.709167 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.709136    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9l2jj"]
Jul 28 19:50:54.713027 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:50:54.713004    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee999ad3_bc3d_4520_83b9_7c1a8f33210b.slice/crio-a2ab9c10ded9a56e26ee578071606b0379bad0afc29edb918eae3504f62cb475 WatchSource:0}: Error finding container a2ab9c10ded9a56e26ee578071606b0379bad0afc29edb918eae3504f62cb475: Status 404 returned error can't find the container with id a2ab9c10ded9a56e26ee578071606b0379bad0afc29edb918eae3504f62cb475
Jul 28 19:50:54.723017 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:54.722992    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:55.411784 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:55.411743    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj" event={"ID":"ee999ad3-bc3d-4520-83b9-7c1a8f33210b","Type":"ContainerStarted","Data":"a2ab9c10ded9a56e26ee578071606b0379bad0afc29edb918eae3504f62cb475"}
Jul 28 19:50:56.478345 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:56.478265    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:56.878779 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:56.878710    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:58.427828 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:58.427797    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj" event={"ID":"ee999ad3-bc3d-4520-83b9-7c1a8f33210b","Type":"ContainerStarted","Data":"61a6678c98e50f4c9a1145cb38b45f12619ae5d96cb58bbf2018a7239939d97b"}
Jul 28 19:50:58.428153 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:58.427838    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj"
Jul 28 19:50:58.442034 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:58.441981    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj" podStartSLOduration=1.074972347 podStartE2EDuration="4.441961045s" totalImagesPullingTime="3.366988698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:50:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:50:58.441667894 +0000 UTC m=+239.622184950" watchObservedRunningTime="2026-07-28 19:50:58.441961045 +0000 UTC m=+239.622478125"
Jul 28 19:50:58.816714 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:58.816639    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:58.917156 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:50:58.917130    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:02.244522 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:02.244484    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:03.880641 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:03.880611    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:04.433779 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:04.433748    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9l2jj"
Jul 28 19:51:05.032677 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.032638    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dr6bc"]
Jul 28 19:51:05.034930 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.034908    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dr6bc"
Jul 28 19:51:05.036676 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.036657    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-clfsq\""
Jul 28 19:51:05.043016 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.042996    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dr6bc"]
Jul 28 19:51:05.109648 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.109621    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86f064ea-5cef-4f79-a80f-6039cfc39581-bound-sa-token\") pod \"cert-manager-55b4d954d5-dr6bc\" (UID: \"86f064ea-5cef-4f79-a80f-6039cfc39581\") " pod="cert-manager/cert-manager-55b4d954d5-dr6bc"
Jul 28 19:51:05.109731 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.109654    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qddhz\" (UniqueName: \"kubernetes.io/projected/86f064ea-5cef-4f79-a80f-6039cfc39581-kube-api-access-qddhz\") pod \"cert-manager-55b4d954d5-dr6bc\" (UID: \"86f064ea-5cef-4f79-a80f-6039cfc39581\") " pod="cert-manager/cert-manager-55b4d954d5-dr6bc"
Jul 28 19:51:05.210810 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.210785    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86f064ea-5cef-4f79-a80f-6039cfc39581-bound-sa-token\") pod \"cert-manager-55b4d954d5-dr6bc\" (UID: \"86f064ea-5cef-4f79-a80f-6039cfc39581\") " pod="cert-manager/cert-manager-55b4d954d5-dr6bc"
Jul 28 19:51:05.210914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.210819    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qddhz\" (UniqueName: \"kubernetes.io/projected/86f064ea-5cef-4f79-a80f-6039cfc39581-kube-api-access-qddhz\") pod \"cert-manager-55b4d954d5-dr6bc\" (UID: \"86f064ea-5cef-4f79-a80f-6039cfc39581\") " pod="cert-manager/cert-manager-55b4d954d5-dr6bc"
Jul 28 19:51:05.223392 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.223373    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86f064ea-5cef-4f79-a80f-6039cfc39581-bound-sa-token\") pod \"cert-manager-55b4d954d5-dr6bc\" (UID: \"86f064ea-5cef-4f79-a80f-6039cfc39581\") " pod="cert-manager/cert-manager-55b4d954d5-dr6bc"
Jul 28 19:51:05.223919 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.223899    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qddhz\" (UniqueName: \"kubernetes.io/projected/86f064ea-5cef-4f79-a80f-6039cfc39581-kube-api-access-qddhz\") pod \"cert-manager-55b4d954d5-dr6bc\" (UID: \"86f064ea-5cef-4f79-a80f-6039cfc39581\") " pod="cert-manager/cert-manager-55b4d954d5-dr6bc"
Jul 28 19:51:05.344824 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.344771    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dr6bc"
Jul 28 19:51:05.469093 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:05.468927    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dr6bc"]
Jul 28 19:51:05.471693 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:51:05.471665    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86f064ea_5cef_4f79_a80f_6039cfc39581.slice/crio-8b45d566e07223ec3ce72ff02258bb3cea0da2b3cc72f7bc630c4817c2f8b722 WatchSource:0}: Error finding container 8b45d566e07223ec3ce72ff02258bb3cea0da2b3cc72f7bc630c4817c2f8b722: Status 404 returned error can't find the container with id 8b45d566e07223ec3ce72ff02258bb3cea0da2b3cc72f7bc630c4817c2f8b722
Jul 28 19:51:06.460436 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:06.460400    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dr6bc" event={"ID":"86f064ea-5cef-4f79-a80f-6039cfc39581","Type":"ContainerStarted","Data":"03424a521a9ca282b0b94ca359bc0fd6e4c172138a63f606e9214dae09431e36"}
Jul 28 19:51:06.460436 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:06.460443    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dr6bc" event={"ID":"86f064ea-5cef-4f79-a80f-6039cfc39581","Type":"ContainerStarted","Data":"8b45d566e07223ec3ce72ff02258bb3cea0da2b3cc72f7bc630c4817c2f8b722"}
Jul 28 19:51:06.474806 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:06.474759    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-dr6bc" podStartSLOduration=1.474745686 podStartE2EDuration="1.474745686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:51:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:51:06.474674054 +0000 UTC m=+247.655191126" watchObservedRunningTime="2026-07-28 19:51:06.474745686 +0000 UTC m=+247.655262742"
Jul 28 19:51:10.438300 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:10.438252    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:10.535907 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:10.535867    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:17.773498 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:17.773452    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:17.876600 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:17.876561    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:21.115260 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:21.115222    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:28.016213 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.016158    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:28.070339 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.070312    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"]
Jul 28 19:51:28.073623 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.073607    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.076746 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.076682    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 19:51:28.076746 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.076707    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:51:28.076746 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.076739    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 19:51:28.076952 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.076776    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-24knk\""
Jul 28 19:51:28.076952 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.076708    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 19:51:28.076952 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.076715    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 19:51:28.082084 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.082004    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"]
Jul 28 19:51:28.093864 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.093833    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:28.189775 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.189750    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/919dcb98-c69d-42f8-8169-b761654a6b3a-metrics-cert\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.189881 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.189788    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l667s\" (UniqueName: \"kubernetes.io/projected/919dcb98-c69d-42f8-8169-b761654a6b3a-kube-api-access-l667s\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.189946 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.189892    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919dcb98-c69d-42f8-8169-b761654a6b3a-cert\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.190000 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.189939    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/919dcb98-c69d-42f8-8169-b761654a6b3a-manager-config\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.291233 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.291174    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/919dcb98-c69d-42f8-8169-b761654a6b3a-metrics-cert\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.291233 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.291203    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l667s\" (UniqueName: \"kubernetes.io/projected/919dcb98-c69d-42f8-8169-b761654a6b3a-kube-api-access-l667s\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.291385 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.291254    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919dcb98-c69d-42f8-8169-b761654a6b3a-cert\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.291385 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.291306    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/919dcb98-c69d-42f8-8169-b761654a6b3a-manager-config\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.291968 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.291942    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/919dcb98-c69d-42f8-8169-b761654a6b3a-manager-config\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.293576 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.293554    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/919dcb98-c69d-42f8-8169-b761654a6b3a-metrics-cert\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.293665 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.293573    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919dcb98-c69d-42f8-8169-b761654a6b3a-cert\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.300347 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.300320    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l667s\" (UniqueName: \"kubernetes.io/projected/919dcb98-c69d-42f8-8169-b761654a6b3a-kube-api-access-l667s\") pod \"lws-controller-manager-6846c7dcc9-58phh\" (UID: \"919dcb98-c69d-42f8-8169-b761654a6b3a\") " pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.385267 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.385247    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:28.510348 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.510320    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"]
Jul 28 19:51:28.513031 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:51:28.513009    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919dcb98_c69d_42f8_8169_b761654a6b3a.slice/crio-c089cca4fe19a620015c65b98643933367eda20f3b217ea3cbd8fc649b2a17d0 WatchSource:0}: Error finding container c089cca4fe19a620015c65b98643933367eda20f3b217ea3cbd8fc649b2a17d0: Status 404 returned error can't find the container with id c089cca4fe19a620015c65b98643933367eda20f3b217ea3cbd8fc649b2a17d0
Jul 28 19:51:28.546482 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:28.546411    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh" event={"ID":"919dcb98-c69d-42f8-8169-b761654a6b3a","Type":"ContainerStarted","Data":"c089cca4fe19a620015c65b98643933367eda20f3b217ea3cbd8fc649b2a17d0"}
Jul 28 19:51:31.635130 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:31.635094    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:31.735938 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:31.735907    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:32.238800 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.238769    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:32.434061 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.434026    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:32.540500 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.540436    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:32.610051 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.610025    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"]
Jul 28 19:51:32.614523 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.614508    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.617463 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.617428    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 19:51:32.617837 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.617822    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-f6pc8\""
Jul 28 19:51:32.617943 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.617926    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 19:51:32.618499 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.618478    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 19:51:32.618608 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.618579    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 19:51:32.637172 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.637152    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"]
Jul 28 19:51:32.730594 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.730570    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-manifests\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.730690 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.730622    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-webhook-cert\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.730690 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.730667    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-tmp\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.730761 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.730705    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6knmw\" (UniqueName: \"kubernetes.io/projected/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-kube-api-access-6knmw\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.730761 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.730753    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.801100 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.801037    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:32.831914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.831885    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6knmw\" (UniqueName: \"kubernetes.io/projected/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-kube-api-access-6knmw\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.832024 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.831956    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.832024 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.831990    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-manifests\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.832093 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.832049    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-webhook-cert\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.832093 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.832072    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-tmp\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.832502 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.832482    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-manifests\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.834411 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.834385    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-tmp\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.834513 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.834498    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.834513 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.834505    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-webhook-cert\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.839766 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.839745    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6knmw\" (UniqueName: \"kubernetes.io/projected/c48574d1-ce18-4c3c-9b12-7828cc8f5df0-kube-api-access-6knmw\") pod \"opendatahub-operator-controller-manager-85967bc8b9-ddpnf\" (UID: \"c48574d1-ce18-4c3c-9b12-7828cc8f5df0\") " pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:32.906291 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.906252    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:32.924824 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:32.924800    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:33.003589 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:33.003562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:33.060483 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:33.060460    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"]
Jul 28 19:51:33.062987 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:51:33.062950    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc48574d1_ce18_4c3c_9b12_7828cc8f5df0.slice/crio-2416ef6e8cf26556803035608e2ef67f4a911af6037d24d4d88d4585767607b4 WatchSource:0}: Error finding container 2416ef6e8cf26556803035608e2ef67f4a911af6037d24d4d88d4585767607b4: Status 404 returned error can't find the container with id 2416ef6e8cf26556803035608e2ef67f4a911af6037d24d4d88d4585767607b4
Jul 28 19:51:33.066635 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:33.066599    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:33.146908 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:33.146884    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:33.247570 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:33.247540    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:33.567977 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:33.567939    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf" event={"ID":"c48574d1-ce18-4c3c-9b12-7828cc8f5df0","Type":"ContainerStarted","Data":"2416ef6e8cf26556803035608e2ef67f4a911af6037d24d4d88d4585767607b4"}
Jul 28 19:51:36.585144 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:36.585063    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf" event={"ID":"c48574d1-ce18-4c3c-9b12-7828cc8f5df0","Type":"ContainerStarted","Data":"dacedfa13617582e87ef140fa2fa5aadfda3ddc86cfe43a5e81747101d5c0f53"}
Jul 28 19:51:37.591420 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:37.591337    2569 generic.go:356] "Generic (PLEG): container finished" podID="c48574d1-ce18-4c3c-9b12-7828cc8f5df0" containerID="dacedfa13617582e87ef140fa2fa5aadfda3ddc86cfe43a5e81747101d5c0f53" exitCode=0
Jul 28 19:51:37.591420 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:37.591390    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf" event={"ID":"c48574d1-ce18-4c3c-9b12-7828cc8f5df0","Type":"ContainerDied","Data":"dacedfa13617582e87ef140fa2fa5aadfda3ddc86cfe43a5e81747101d5c0f53"}
Jul 28 19:51:38.598757 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:38.598716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf" event={"ID":"c48574d1-ce18-4c3c-9b12-7828cc8f5df0","Type":"ContainerStarted","Data":"347af7bfcee3ff868f009f0491ad68e19a0b310edad3009cfcf77bf5783ba314"}
Jul 28 19:51:38.599144 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:38.598842    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:38.620260 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:38.620200    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf" podStartSLOduration=1.980795702 podStartE2EDuration="6.62017918s" totalImagesPullingTime="3.622192499s" totalInitContainerRuntime="1.017190979s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:51:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:51:38.617645847 +0000 UTC m=+279.798162903" watchObservedRunningTime="2026-07-28 19:51:38.62017918 +0000 UTC m=+279.800696237"
Jul 28 19:51:40.343369 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:40.343331    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:40.397291 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:40.397249    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:40.609888 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:40.609803    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh" event={"ID":"919dcb98-c69d-42f8-8169-b761654a6b3a","Type":"ContainerStarted","Data":"fb8632be1e009a5a92f9362ce1f4a64332d074cb271c534042a2ab75db2e3b5a"}
Jul 28 19:51:40.610036 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:40.609914    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:40.626180 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:40.626136    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh" podStartSLOduration=1.439297961 podStartE2EDuration="12.626119529s" totalImagesPullingTime="11.186821568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:51:40.62443594 +0000 UTC m=+281.804952997" watchObservedRunningTime="2026-07-28 19:51:40.626119529 +0000 UTC m=+281.806636586"
Jul 28 19:51:49.606183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:49.606155    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85967bc8b9-ddpnf"
Jul 28 19:51:51.617291 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:51.617246    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6846c7dcc9-58phh"
Jul 28 19:51:51.916659 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:51.916577    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:52.044694 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.044660    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:52.424945 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.424910    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"]
Jul 28 19:51:52.427417 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.427393    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"
Jul 28 19:51:52.429926 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.429903    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 19:51:52.430296 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.430264    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-c6ssc\""
Jul 28 19:51:52.430439 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.430418    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 19:51:52.430439 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.430429    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 19:51:52.430612 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.430292    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 19:51:52.438811 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.438791    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"]
Jul 28 19:51:52.596660 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.596631    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/281eb0aa-e5cf-4071-a6bf-de2fa275ed85-tmp\") pod \"kube-auth-proxy-579d8dfb47-bbhf8\" (UID: \"281eb0aa-e5cf-4071-a6bf-de2fa275ed85\") " pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"
Jul 28 19:51:52.596801 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.596667    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgfps\" (UniqueName: \"kubernetes.io/projected/281eb0aa-e5cf-4071-a6bf-de2fa275ed85-kube-api-access-hgfps\") pod \"kube-auth-proxy-579d8dfb47-bbhf8\" (UID: \"281eb0aa-e5cf-4071-a6bf-de2fa275ed85\") " pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"
Jul 28 19:51:52.596801 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.596775    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/281eb0aa-e5cf-4071-a6bf-de2fa275ed85-tls-certs\") pod \"kube-auth-proxy-579d8dfb47-bbhf8\" (UID: \"281eb0aa-e5cf-4071-a6bf-de2fa275ed85\") " pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"
Jul 28 19:51:52.697896 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.697834    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/281eb0aa-e5cf-4071-a6bf-de2fa275ed85-tls-certs\") pod \"kube-auth-proxy-579d8dfb47-bbhf8\" (UID: \"281eb0aa-e5cf-4071-a6bf-de2fa275ed85\") " pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"
Jul 28 19:51:52.697896 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.697888    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/281eb0aa-e5cf-4071-a6bf-de2fa275ed85-tmp\") pod \"kube-auth-proxy-579d8dfb47-bbhf8\" (UID: \"281eb0aa-e5cf-4071-a6bf-de2fa275ed85\") " pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"
Jul 28 19:51:52.698263 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.698054    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgfps\" (UniqueName: \"kubernetes.io/projected/281eb0aa-e5cf-4071-a6bf-de2fa275ed85-kube-api-access-hgfps\") pod \"kube-auth-proxy-579d8dfb47-bbhf8\" (UID: \"281eb0aa-e5cf-4071-a6bf-de2fa275ed85\") " pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"
Jul 28 19:51:52.700172 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.700148    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/281eb0aa-e5cf-4071-a6bf-de2fa275ed85-tmp\") pod \"kube-auth-proxy-579d8dfb47-bbhf8\" (UID: \"281eb0aa-e5cf-4071-a6bf-de2fa275ed85\") " pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"
Jul 28 19:51:52.700383 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.700365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/281eb0aa-e5cf-4071-a6bf-de2fa275ed85-tls-certs\") pod \"kube-auth-proxy-579d8dfb47-bbhf8\" (UID: \"281eb0aa-e5cf-4071-a6bf-de2fa275ed85\") " pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"
Jul 28 19:51:52.732006 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.731986    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgfps\" (UniqueName: \"kubernetes.io/projected/281eb0aa-e5cf-4071-a6bf-de2fa275ed85-kube-api-access-hgfps\") pod \"kube-auth-proxy-579d8dfb47-bbhf8\" (UID: \"281eb0aa-e5cf-4071-a6bf-de2fa275ed85\") " pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"
Jul 28 19:51:52.738811 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.738792    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"
Jul 28 19:51:52.889956 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:52.889794    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8"]
Jul 28 19:51:52.894752 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:51:52.894698    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod281eb0aa_e5cf_4071_a6bf_de2fa275ed85.slice/crio-c6d67f89ccd8e42e7f5edd2b3766a5aa7158964f102925556ef0443efe019e35 WatchSource:0}: Error finding container c6d67f89ccd8e42e7f5edd2b3766a5aa7158964f102925556ef0443efe019e35: Status 404 returned error can't find the container with id c6d67f89ccd8e42e7f5edd2b3766a5aa7158964f102925556ef0443efe019e35
Jul 28 19:51:53.548022 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:53.547984    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:53.671781 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:53.671740    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8" event={"ID":"281eb0aa-e5cf-4071-a6bf-de2fa275ed85","Type":"ContainerStarted","Data":"c6d67f89ccd8e42e7f5edd2b3766a5aa7158964f102925556ef0443efe019e35"}
Jul 28 19:51:53.683978 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:53.683949    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:53.807387 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:53.807259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:53.870354 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:53.870318    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:53.935095 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:53.935019    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:54.821301 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:54.821244    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:51:55.683362 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:55.683323    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8" event={"ID":"281eb0aa-e5cf-4071-a6bf-de2fa275ed85","Type":"ContainerStarted","Data":"4eb2fc91ae1a1e6e25899669d2747633b018c54a909eb6eab81ead133001de9c"}
Jul 28 19:51:55.698616 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:55.698567    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-579d8dfb47-bbhf8" podStartSLOduration=1.056179935 podStartE2EDuration="3.698552379s" totalImagesPullingTime="2.642372444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:51:55.697342573 +0000 UTC m=+296.877859628" watchObservedRunningTime="2026-07-28 19:51:55.698552379 +0000 UTC m=+296.879069435"
Jul 28 19:51:59.197213 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:59.197187    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 19:51:59.197807 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:59.197262    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 19:51:59.203146 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:59.203117    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 19:51:59.938165 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:51:59.938130    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:00.057442 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:00.057392    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:00.229157 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:00.229082    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:00.373817 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:00.373784    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:00.561109 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:00.561044    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:02.605513 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.605475    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:02.659214 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.659179    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"]
Jul 28 19:52:02.662641 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.662621    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"
Jul 28 19:52:02.664574 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.664546    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 19:52:02.664669 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.664579    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pzssz\""
Jul 28 19:52:02.672360 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.672337    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"]
Jul 28 19:52:02.688193 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.688167    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d38055c-6d25-4719-b1ce-b49781a2c73c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ddmnl\" (UID: \"8d38055c-6d25-4719-b1ce-b49781a2c73c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"
Jul 28 19:52:02.688358 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.688239    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qdfs\" (UniqueName: \"kubernetes.io/projected/8d38055c-6d25-4719-b1ce-b49781a2c73c-kube-api-access-4qdfs\") pod \"llmisvc-controller-manager-777c6bd7d5-ddmnl\" (UID: \"8d38055c-6d25-4719-b1ce-b49781a2c73c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"
Jul 28 19:52:02.691507 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.691477    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:02.789585 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.789557    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qdfs\" (UniqueName: \"kubernetes.io/projected/8d38055c-6d25-4719-b1ce-b49781a2c73c-kube-api-access-4qdfs\") pod \"llmisvc-controller-manager-777c6bd7d5-ddmnl\" (UID: \"8d38055c-6d25-4719-b1ce-b49781a2c73c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"
Jul 28 19:52:02.789770 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.789636    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d38055c-6d25-4719-b1ce-b49781a2c73c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ddmnl\" (UID: \"8d38055c-6d25-4719-b1ce-b49781a2c73c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"
Jul 28 19:52:02.789770 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:52:02.789733    2569 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 28 19:52:02.789899 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:52:02.789791    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d38055c-6d25-4719-b1ce-b49781a2c73c-cert podName:8d38055c-6d25-4719-b1ce-b49781a2c73c nodeName:}" failed. No retries permitted until 2026-07-28 19:52:03.289775291 +0000 UTC m=+304.470292325 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8d38055c-6d25-4719-b1ce-b49781a2c73c-cert") pod "llmisvc-controller-manager-777c6bd7d5-ddmnl" (UID: "8d38055c-6d25-4719-b1ce-b49781a2c73c") : secret "llmisvc-webhook-server-cert" not found
Jul 28 19:52:02.806107 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:02.806079    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qdfs\" (UniqueName: \"kubernetes.io/projected/8d38055c-6d25-4719-b1ce-b49781a2c73c-kube-api-access-4qdfs\") pod \"llmisvc-controller-manager-777c6bd7d5-ddmnl\" (UID: \"8d38055c-6d25-4719-b1ce-b49781a2c73c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"
Jul 28 19:52:03.022448 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:03.022421    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:03.118137 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:03.118107    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:03.293602 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:03.293517    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d38055c-6d25-4719-b1ce-b49781a2c73c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ddmnl\" (UID: \"8d38055c-6d25-4719-b1ce-b49781a2c73c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"
Jul 28 19:52:03.296170 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:03.296138    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d38055c-6d25-4719-b1ce-b49781a2c73c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ddmnl\" (UID: \"8d38055c-6d25-4719-b1ce-b49781a2c73c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"
Jul 28 19:52:03.573424 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:03.573339    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"
Jul 28 19:52:03.715839 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:03.715815    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"]
Jul 28 19:52:03.718748 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:52:03.718352    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8d38055c_6d25_4719_b1ce_b49781a2c73c.slice/crio-c49df2477a23763e6a8252d42da27dd520cb023f48602b5bc83252373632a277 WatchSource:0}: Error finding container c49df2477a23763e6a8252d42da27dd520cb023f48602b5bc83252373632a277: Status 404 returned error can't find the container with id c49df2477a23763e6a8252d42da27dd520cb023f48602b5bc83252373632a277
Jul 28 19:52:03.720502 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:03.720460    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:52:04.045845 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:04.045815    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:04.114714 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:04.114686    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:04.728151 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:04.728107    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl" event={"ID":"8d38055c-6d25-4719-b1ce-b49781a2c73c","Type":"ContainerStarted","Data":"c49df2477a23763e6a8252d42da27dd520cb023f48602b5bc83252373632a277"}
Jul 28 19:52:05.947500 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:05.947463    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:06.049303 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:06.049253    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:07.746984 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:07.746948    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl" event={"ID":"8d38055c-6d25-4719-b1ce-b49781a2c73c","Type":"ContainerStarted","Data":"ba72fbdfca39cb55488e904a1332f0de8c489ba03c554a455e0978ac911f392a"}
Jul 28 19:52:07.747470 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:07.747071    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"
Jul 28 19:52:07.762326 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:07.762254    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl" podStartSLOduration=2.654628361 podStartE2EDuration="5.76223775s" totalImagesPullingTime="3.107609389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:52:07.761357741 +0000 UTC m=+308.941874798" watchObservedRunningTime="2026-07-28 19:52:07.76223775 +0000 UTC m=+308.942754808"
Jul 28 19:52:33.919568 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:33.919533    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:34.009886 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:34.009858    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:36.102843 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:36.102805    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:36.205468 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:36.205435    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:38.137923 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:38.137862    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:38.234980 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:38.234954    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:38.754621 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:38.754593    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddmnl"
Jul 28 19:52:53.577240 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:53.577170    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:53.685388 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:53.685347    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:53.789731 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:53.789702    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:54.485160 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:54.485124    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:52:55.268700 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:52:55.268652    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:21.506629 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:21.506596    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:21.574752 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:21.574719    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:23.167629 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:23.167595    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:26.615033 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:26.614998    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:28.550672 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.550637    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:28.669726 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.669700    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c7794ffb5-jt8bp"]
Jul 28 19:53:28.672962 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.672939    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.675147 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.675105    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 19:53:28.675522 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.675492    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 19:53:28.675675 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.675646    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 19:53:28.675827 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.675808    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lxsh7\""
Jul 28 19:53:28.676022 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.676008    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 19:53:28.676104 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.676088    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 19:53:28.676330 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.676310    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 19:53:28.677664 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.677646    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 19:53:28.682416 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.682390    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 19:53:28.684323 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.684305    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c7794ffb5-jt8bp"]
Jul 28 19:53:28.744892 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.744872    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:28.804115 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.804062    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-console-config\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.804115 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.804106    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-oauth-serving-cert\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.804230 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.804131    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnhkg\" (UniqueName: \"kubernetes.io/projected/08e3d059-8f02-43f7-af82-3fe2345a6cfa-kube-api-access-xnhkg\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.804230 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.804190    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08e3d059-8f02-43f7-af82-3fe2345a6cfa-console-oauth-config\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.804230 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.804217    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-trusted-ca-bundle\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.804385 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.804269    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08e3d059-8f02-43f7-af82-3fe2345a6cfa-console-serving-cert\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.804385 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.804356    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-service-ca\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.905125 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.905103    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-oauth-serving-cert\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.905228 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.905133    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhkg\" (UniqueName: \"kubernetes.io/projected/08e3d059-8f02-43f7-af82-3fe2345a6cfa-kube-api-access-xnhkg\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.905228 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.905154    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08e3d059-8f02-43f7-af82-3fe2345a6cfa-console-oauth-config\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.905228 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.905169    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-trusted-ca-bundle\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.905419 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.905229    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08e3d059-8f02-43f7-af82-3fe2345a6cfa-console-serving-cert\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.905419 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.905261    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-service-ca\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.905419 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.905407    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-console-config\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.906073 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.906052    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-trusted-ca-bundle\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.906174 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.906110    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-oauth-serving-cert\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.906174 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.906145    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-console-config\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.906174 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.906148    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08e3d059-8f02-43f7-af82-3fe2345a6cfa-service-ca\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.907768 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.907750    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08e3d059-8f02-43f7-af82-3fe2345a6cfa-console-serving-cert\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.907848 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.907771    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08e3d059-8f02-43f7-af82-3fe2345a6cfa-console-oauth-config\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.919016 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.918998    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnhkg\" (UniqueName: \"kubernetes.io/projected/08e3d059-8f02-43f7-af82-3fe2345a6cfa-kube-api-access-xnhkg\") pod \"console-7c7794ffb5-jt8bp\" (UID: \"08e3d059-8f02-43f7-af82-3fe2345a6cfa\") " pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:28.982876 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:28.982853    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:29.120008 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:29.119979    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c7794ffb5-jt8bp"]
Jul 28 19:53:29.123044 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:53:29.123018    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e3d059_8f02_43f7_af82_3fe2345a6cfa.slice/crio-a41cbbef11a49e4357950f7f4364f3f3adae5f289dc03f654d24b40764fc4c7e WatchSource:0}: Error finding container a41cbbef11a49e4357950f7f4364f3f3adae5f289dc03f654d24b40764fc4c7e: Status 404 returned error can't find the container with id a41cbbef11a49e4357950f7f4364f3f3adae5f289dc03f654d24b40764fc4c7e
Jul 28 19:53:30.120451 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:30.120420    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7794ffb5-jt8bp" event={"ID":"08e3d059-8f02-43f7-af82-3fe2345a6cfa","Type":"ContainerStarted","Data":"7b6e9fead81bb40d9c7ad40ccf2d903b51965f7e7df04c1bd4bcd30d6bffb570"}
Jul 28 19:53:30.120822 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:30.120468    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7794ffb5-jt8bp" event={"ID":"08e3d059-8f02-43f7-af82-3fe2345a6cfa","Type":"ContainerStarted","Data":"a41cbbef11a49e4357950f7f4364f3f3adae5f289dc03f654d24b40764fc4c7e"}
Jul 28 19:53:30.126617 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:30.126597    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:30.143645 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:30.143601    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c7794ffb5-jt8bp" podStartSLOduration=2.143586537 podStartE2EDuration="2.143586537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:53:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:53:30.142242206 +0000 UTC m=+391.322759262" watchObservedRunningTime="2026-07-28 19:53:30.143586537 +0000 UTC m=+391.324103571"
Jul 28 19:53:30.394885 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:30.394805    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:30.537217 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:30.537187    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:33.754469 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:33.754437    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:33.997859 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:33.997822    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:36.834749 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:36.834720    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:37.039043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:37.039008    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:37.129228 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:37.129163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:38.345556 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:38.345522    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:38.420786 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:38.420757    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:38.983908 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:38.983879    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:38.984069 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:38.983925    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:38.988230 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:38.988210    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:39.164978 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:39.164954    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c7794ffb5-jt8bp"
Jul 28 19:53:39.328804 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:39.328731    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:39.429928 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:39.429901    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:39.530415 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:39.530389    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:46.134073 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:46.134040    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:46.268576 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:46.268539    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:48.306916 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.306885    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"]
Jul 28 19:53:48.310427 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.310411    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 19:53:48.315807 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.315785    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 19:53:48.315807 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.315794    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 19:53:48.315959 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.315859    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nnmnz\""
Jul 28 19:53:48.344606 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.344584    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"]
Jul 28 19:53:48.365084 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.365061    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03ff54f8-6c19-4448-86e7-6ef855f164e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kqkf8\" (UID: \"03ff54f8-6c19-4448-86e7-6ef855f164e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 19:53:48.365184 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.365114    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m722q\" (UniqueName: \"kubernetes.io/projected/03ff54f8-6c19-4448-86e7-6ef855f164e1-kube-api-access-m722q\") pod \"kuadrant-operator-controller-manager-86878c769f-kqkf8\" (UID: \"03ff54f8-6c19-4448-86e7-6ef855f164e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 19:53:48.465807 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.465783    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m722q\" (UniqueName: \"kubernetes.io/projected/03ff54f8-6c19-4448-86e7-6ef855f164e1-kube-api-access-m722q\") pod \"kuadrant-operator-controller-manager-86878c769f-kqkf8\" (UID: \"03ff54f8-6c19-4448-86e7-6ef855f164e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 19:53:48.465942 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.465840    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03ff54f8-6c19-4448-86e7-6ef855f164e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kqkf8\" (UID: \"03ff54f8-6c19-4448-86e7-6ef855f164e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 19:53:48.466221 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.466202    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03ff54f8-6c19-4448-86e7-6ef855f164e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kqkf8\" (UID: \"03ff54f8-6c19-4448-86e7-6ef855f164e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 19:53:48.484242 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.484215    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m722q\" (UniqueName: \"kubernetes.io/projected/03ff54f8-6c19-4448-86e7-6ef855f164e1-kube-api-access-m722q\") pod \"kuadrant-operator-controller-manager-86878c769f-kqkf8\" (UID: \"03ff54f8-6c19-4448-86e7-6ef855f164e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 19:53:48.621484 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.621386    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 19:53:48.781563 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:48.781534    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"]
Jul 28 19:53:49.210938 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:49.210907    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8" event={"ID":"03ff54f8-6c19-4448-86e7-6ef855f164e1","Type":"ContainerStarted","Data":"656ca981fcc065785c97a292711d774ae1abdc56a42dc7871f5e674e536df1e9"}
Jul 28 19:53:51.765445 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:51.765407    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:51.864910 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:51.864874    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:53.233182 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:53.233147    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8" event={"ID":"03ff54f8-6c19-4448-86e7-6ef855f164e1","Type":"ContainerStarted","Data":"7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922"}
Jul 28 19:53:53.233594 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:53.233228    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 19:53:53.252147 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:53.252106    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8" podStartSLOduration=1.353149578 podStartE2EDuration="5.252093274s" totalImagesPullingTime="3.898943696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:53:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:53:53.251476757 +0000 UTC m=+414.431993813" watchObservedRunningTime="2026-07-28 19:53:53.252093274 +0000 UTC m=+414.432610330"
Jul 28 19:53:54.767783 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:54.767747    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:54.871021 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:54.870990    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:58.568477 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:58.568435    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:53:58.628311 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:53:58.628259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:03.811251 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:03.811214    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:04.240106 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:04.240069    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 19:54:04.953106 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:04.953077    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:05.053990 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:05.053957    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:11.491116 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:11.491081    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:11.590941 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:11.590901    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:14.494136 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:14.494094    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:14.594913 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:14.594884    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:18.494375 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:18.494336    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:18.594697 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:18.594668    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:20.728564 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:20.728531    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:21.010501 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:21.010411    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:23.241798 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:23.241739    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:23.359981 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:23.359949    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:23.424736 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:23.424698    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:24.095578 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:24.095544    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:24.162145 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:24.162114    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:26.292598 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:26.292556    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:26.373757 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:26.373731    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:33.093766 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:33.093727    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:33.193629 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:33.193595    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:33.893022 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:33.892989    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:33.946359 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:33.946329    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:37.696964 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:37.696928    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:37.796980 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:37.796947    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:44.633314 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:44.633261    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:44.846800 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:44.846759    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:45.170873 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:45.170840    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:57.698734 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:57.698701    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:57.990187 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:57.990119    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:58.468527 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:54:58.468494    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:05.395212 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.395175    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:05.489237 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.489203    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:05.543415 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.543383    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:05.603374 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.603342    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b5686dc45-msscs"]
Jul 28 19:55:05.606018 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.605997    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:55:05.607765 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.607748    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 19:55:05.607867 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.607748    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 19:55:05.607867 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.607800    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kbtxl\""
Jul 28 19:55:05.616547 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.616528    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-msscs"]
Jul 28 19:55:05.708873 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.708853    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlnpw\" (UniqueName: \"kubernetes.io/projected/22c541b8-e3ec-49b4-a002-1bad095243e4-kube-api-access-hlnpw\") pod \"maas-controller-5b5686dc45-msscs\" (UID: \"22c541b8-e3ec-49b4-a002-1bad095243e4\") " pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:55:05.708973 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.708884    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/22c541b8-e3ec-49b4-a002-1bad095243e4-webhook-cert\") pod \"maas-controller-5b5686dc45-msscs\" (UID: \"22c541b8-e3ec-49b4-a002-1bad095243e4\") " pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:55:05.773432 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.773407    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:05.809527 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.809506    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlnpw\" (UniqueName: \"kubernetes.io/projected/22c541b8-e3ec-49b4-a002-1bad095243e4-kube-api-access-hlnpw\") pod \"maas-controller-5b5686dc45-msscs\" (UID: \"22c541b8-e3ec-49b4-a002-1bad095243e4\") " pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:55:05.809615 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.809534    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/22c541b8-e3ec-49b4-a002-1bad095243e4-webhook-cert\") pod \"maas-controller-5b5686dc45-msscs\" (UID: \"22c541b8-e3ec-49b4-a002-1bad095243e4\") " pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:55:05.811901 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.811882    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/22c541b8-e3ec-49b4-a002-1bad095243e4-webhook-cert\") pod \"maas-controller-5b5686dc45-msscs\" (UID: \"22c541b8-e3ec-49b4-a002-1bad095243e4\") " pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:55:05.817183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.817158    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlnpw\" (UniqueName: \"kubernetes.io/projected/22c541b8-e3ec-49b4-a002-1bad095243e4-kube-api-access-hlnpw\") pod \"maas-controller-5b5686dc45-msscs\" (UID: \"22c541b8-e3ec-49b4-a002-1bad095243e4\") " pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:55:05.916777 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:05.916750    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:55:06.045346 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:06.045324    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-msscs"]
Jul 28 19:55:06.047466 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:55:06.047437    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22c541b8_e3ec_49b4_a002_1bad095243e4.slice/crio-de681c220f8e6d08c7ea9dce6bedd0c5919455d5549a61bedb1a9f8f7b2d5764 WatchSource:0}: Error finding container de681c220f8e6d08c7ea9dce6bedd0c5919455d5549a61bedb1a9f8f7b2d5764: Status 404 returned error can't find the container with id de681c220f8e6d08c7ea9dce6bedd0c5919455d5549a61bedb1a9f8f7b2d5764
Jul 28 19:55:06.566136 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:06.566096    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5686dc45-msscs" event={"ID":"22c541b8-e3ec-49b4-a002-1bad095243e4","Type":"ContainerStarted","Data":"de681c220f8e6d08c7ea9dce6bedd0c5919455d5549a61bedb1a9f8f7b2d5764"}
Jul 28 19:55:08.385481 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:08.385438    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:08.583043 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:08.582955    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5686dc45-msscs" event={"ID":"22c541b8-e3ec-49b4-a002-1bad095243e4","Type":"ContainerStarted","Data":"285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929"}
Jul 28 19:55:08.583208 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:08.583098    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:55:08.599441 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:08.599396    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b5686dc45-msscs" podStartSLOduration=1.411626823 podStartE2EDuration="3.599382621s" totalImagesPullingTime="2.187755798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:55:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:55:08.59839365 +0000 UTC m=+489.778910705" watchObservedRunningTime="2026-07-28 19:55:08.599382621 +0000 UTC m=+489.779899676"
Jul 28 19:55:08.722635 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:08.722604    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:19.590263 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:19.590232    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:55:38.020407 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:38.020360    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:38.176485 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:38.176446    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:38.595298 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:38.595256    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:38.827590 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:38.827561    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:39.019987 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:39.019956    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:39.115545 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:39.115519    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:49.745813 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:49.745770    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:55:49.845107 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:55:49.845077    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:09.770054 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.769973    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"]
Jul 28 19:56:09.774344 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.774322    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.776431 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.776404    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f8572\""
Jul 28 19:56:09.776561 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.776454    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 19:56:09.776561 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.776482    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 28 19:56:09.776561 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.776495    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 19:56:09.781324 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.781305    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"]
Jul 28 19:56:09.799809 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.799782    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:09.823810 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.823782    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.823916 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.823824    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.823916 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.823857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfqqk\" (UniqueName: \"kubernetes.io/projected/932bb7be-5749-49be-9779-bf91092722f7-kube-api-access-bfqqk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.824030 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.823922    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.824030 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.823991    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/932bb7be-5749-49be-9779-bf91092722f7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.824132 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.824028    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.924876 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.924847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfqqk\" (UniqueName: \"kubernetes.io/projected/932bb7be-5749-49be-9779-bf91092722f7-kube-api-access-bfqqk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.925004 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.924891    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.925004 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.924926    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/932bb7be-5749-49be-9779-bf91092722f7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.925004 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.924965    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.925004 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.925000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.925222 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.925040    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.925356 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.925335    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.925434 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.925411    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.925476 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.925441    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.927104 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.927084    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/932bb7be-5749-49be-9779-bf91092722f7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.927380 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.927362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/932bb7be-5749-49be-9779-bf91092722f7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:09.931620 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:09.931593    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfqqk\" (UniqueName: \"kubernetes.io/projected/932bb7be-5749-49be-9779-bf91092722f7-kube-api-access-bfqqk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p\" (UID: \"932bb7be-5749-49be-9779-bf91092722f7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:10.086400 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:10.086330    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:10.217469 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:10.217439    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"]
Jul 28 19:56:10.219080 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:56:10.219053    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod932bb7be_5749_49be_9779_bf91092722f7.slice/crio-645cfa3f6682a585aef5f462fe651d82899776050ea0d5610f50c91cb8f54211 WatchSource:0}: Error finding container 645cfa3f6682a585aef5f462fe651d82899776050ea0d5610f50c91cb8f54211: Status 404 returned error can't find the container with id 645cfa3f6682a585aef5f462fe651d82899776050ea0d5610f50c91cb8f54211
Jul 28 19:56:10.880622 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:10.880575    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p" event={"ID":"932bb7be-5749-49be-9779-bf91092722f7","Type":"ContainerStarted","Data":"645cfa3f6682a585aef5f462fe651d82899776050ea0d5610f50c91cb8f54211"}
Jul 28 19:56:11.887222 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:11.887177    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p" event={"ID":"932bb7be-5749-49be-9779-bf91092722f7","Type":"ContainerStarted","Data":"ac4020a55e2d63a1227197f4afd39bf3be2667928d0ad95e42740d88eefc5e1f"}
Jul 28 19:56:11.905834 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:11.905779    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p" podStartSLOduration=1.735963157 podStartE2EDuration="2.9057648s" totalImagesPullingTime="1.169801643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:56:11.904032684 +0000 UTC m=+553.084549741" watchObservedRunningTime="2026-07-28 19:56:11.9057648 +0000 UTC m=+553.086281856"
Jul 28 19:56:20.087193 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:20.087154    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:20.087700 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:20.087210    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:20.099913 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:20.099885    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:20.939734 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:20.939706    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p"
Jul 28 19:56:22.906492 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:22.906452    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:38.198923 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:38.198892    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:43.971193 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:43.971137    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"]
Jul 28 19:56:43.976431 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:43.976411    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:43.978759 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:43.978738    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 28 19:56:43.983723 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:43.983700    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"]
Jul 28 19:56:44.000136 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.000103    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:44.103662 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.103629    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f2756a3-16e9-4d20-9782-2f49e4e728dc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.103794 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.103676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.103794 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.103744    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tngh6\" (UniqueName: \"kubernetes.io/projected/1f2756a3-16e9-4d20-9782-2f49e4e728dc-kube-api-access-tngh6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.103794 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.103772    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.103978 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.103803    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.103978 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.103905    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.204793 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.204762    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.204925 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.204813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tngh6\" (UniqueName: \"kubernetes.io/projected/1f2756a3-16e9-4d20-9782-2f49e4e728dc-kube-api-access-tngh6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.204925 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.204860    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.204925 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.204892    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.205091 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.204976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.205091 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.205029    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f2756a3-16e9-4d20-9782-2f49e4e728dc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.205196 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.205165    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.205332 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.205306    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.205448 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.205379    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.206965 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.206938    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f2756a3-16e9-4d20-9782-2f49e4e728dc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.207272 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.207252    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f2756a3-16e9-4d20-9782-2f49e4e728dc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.212699 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.212673    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tngh6\" (UniqueName: \"kubernetes.io/projected/1f2756a3-16e9-4d20-9782-2f49e4e728dc-kube-api-access-tngh6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2\" (UID: \"1f2756a3-16e9-4d20-9782-2f49e4e728dc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.289014 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.288959    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:44.418848 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:44.418823    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"]
Jul 28 19:56:44.421489 ip-10-0-134-138 kubenswrapper[2569]: W0728 19:56:44.421461    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f2756a3_16e9_4d20_9782_2f49e4e728dc.slice/crio-84f2448d5665828f4c4f3042951393e5a9c89cd105d1a60532a89d509e5ecd5f WatchSource:0}: Error finding container 84f2448d5665828f4c4f3042951393e5a9c89cd105d1a60532a89d509e5ecd5f: Status 404 returned error can't find the container with id 84f2448d5665828f4c4f3042951393e5a9c89cd105d1a60532a89d509e5ecd5f
Jul 28 19:56:45.047626 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:45.047580    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2" event={"ID":"1f2756a3-16e9-4d20-9782-2f49e4e728dc","Type":"ContainerStarted","Data":"4cde8cf5dd4a925f41b6c1ff2e5468df7be6f3271ff55198c316d77116a90255"}
Jul 28 19:56:45.048026 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:45.047635    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2" event={"ID":"1f2756a3-16e9-4d20-9782-2f49e4e728dc","Type":"ContainerStarted","Data":"84f2448d5665828f4c4f3042951393e5a9c89cd105d1a60532a89d509e5ecd5f"}
Jul 28 19:56:45.063824 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:45.063779    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2" podStartSLOduration=1.849232414 podStartE2EDuration="2.063764212s" totalImagesPullingTime="214.531798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:56:45.062512151 +0000 UTC m=+586.243029204" watchObservedRunningTime="2026-07-28 19:56:45.063764212 +0000 UTC m=+586.244281267"
Jul 28 19:56:51.590183 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:51.590149    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:56:54.289187 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:54.289147    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:54.289595 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:54.289231    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:54.301927 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:54.301896    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:55.108917 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:55.108890    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2"
Jul 28 19:56:59.227087 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:59.227060    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 19:56:59.229621 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:56:59.229589    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 19:57:11.962956 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:57:11.962922    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:21.000729 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:57:21.000694    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:52.495472 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:57:52.495436    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:57:52.593914 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:57:52.593884    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:10.156758 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:58:10.156713    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:15.951079 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:58:15.951041    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:16.053264 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:58:16.053234    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:17.135500 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:58:17.135467    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:17.225305 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:58:17.225257    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:55.452323 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:58:55.452226    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 19:59:03.849593 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:03.849559    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-msscs"]
Jul 28 19:59:03.849995 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:03.849755    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b5686dc45-msscs" podUID="22c541b8-e3ec-49b4-a002-1bad095243e4" containerName="manager" containerID="cri-o://285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929" gracePeriod=30
Jul 28 19:59:04.103030 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.102968    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:59:04.184086 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.184048    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/22c541b8-e3ec-49b4-a002-1bad095243e4-webhook-cert\") pod \"22c541b8-e3ec-49b4-a002-1bad095243e4\" (UID: \"22c541b8-e3ec-49b4-a002-1bad095243e4\") "
Jul 28 19:59:04.184221 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.184108    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlnpw\" (UniqueName: \"kubernetes.io/projected/22c541b8-e3ec-49b4-a002-1bad095243e4-kube-api-access-hlnpw\") pod \"22c541b8-e3ec-49b4-a002-1bad095243e4\" (UID: \"22c541b8-e3ec-49b4-a002-1bad095243e4\") "
Jul 28 19:59:04.186097 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.186061    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c541b8-e3ec-49b4-a002-1bad095243e4-kube-api-access-hlnpw" (OuterVolumeSpecName: "kube-api-access-hlnpw") pod "22c541b8-e3ec-49b4-a002-1bad095243e4" (UID: "22c541b8-e3ec-49b4-a002-1bad095243e4"). InnerVolumeSpecName "kube-api-access-hlnpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:59:04.186202 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.186089    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22c541b8-e3ec-49b4-a002-1bad095243e4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "22c541b8-e3ec-49b4-a002-1bad095243e4" (UID: "22c541b8-e3ec-49b4-a002-1bad095243e4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:59:04.285410 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.285380    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/22c541b8-e3ec-49b4-a002-1bad095243e4-webhook-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 19:59:04.285410 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.285408    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlnpw\" (UniqueName: \"kubernetes.io/projected/22c541b8-e3ec-49b4-a002-1bad095243e4-kube-api-access-hlnpw\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 19:59:04.711088 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.711056    2569 generic.go:356] "Generic (PLEG): container finished" podID="22c541b8-e3ec-49b4-a002-1bad095243e4" containerID="285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929" exitCode=0
Jul 28 19:59:04.711255 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.711109    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5686dc45-msscs" event={"ID":"22c541b8-e3ec-49b4-a002-1bad095243e4","Type":"ContainerDied","Data":"285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929"}
Jul 28 19:59:04.711255 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.711126    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5686dc45-msscs"
Jul 28 19:59:04.711255 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.711149    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5686dc45-msscs" event={"ID":"22c541b8-e3ec-49b4-a002-1bad095243e4","Type":"ContainerDied","Data":"de681c220f8e6d08c7ea9dce6bedd0c5919455d5549a61bedb1a9f8f7b2d5764"}
Jul 28 19:59:04.711255 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.711170    2569 scope.go:117] "RemoveContainer" containerID="285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929"
Jul 28 19:59:04.720902 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.720886    2569 scope.go:117] "RemoveContainer" containerID="285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929"
Jul 28 19:59:04.721153 ip-10-0-134-138 kubenswrapper[2569]: E0728 19:59:04.721128    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929\": container with ID starting with 285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929 not found: ID does not exist" containerID="285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929"
Jul 28 19:59:04.721215 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.721165    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929"} err="failed to get container status \"285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929\": rpc error: code = NotFound desc = could not find container \"285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929\": container with ID starting with 285f070f7d256458a05f7f3680f5beaab367ae04bb0cd2b06fe3110072d6d929 not found: ID does not exist"
Jul 28 19:59:04.733106 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.733077    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-msscs"]
Jul 28 19:59:04.734829 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:04.734806    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-msscs"]
Jul 28 19:59:05.366510 ip-10-0-134-138 kubenswrapper[2569]: I0728 19:59:05.366478    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22c541b8-e3ec-49b4-a002-1bad095243e4" path="/var/lib/kubelet/pods/22c541b8-e3ec-49b4-a002-1bad095243e4/volumes"
Jul 28 20:00:05.525299 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.525241    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b5686dc45-gn8nb"]
Jul 28 20:00:05.525707 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.525632    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22c541b8-e3ec-49b4-a002-1bad095243e4" containerName="manager"
Jul 28 20:00:05.525707 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.525643    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="22c541b8-e3ec-49b4-a002-1bad095243e4" containerName="manager"
Jul 28 20:00:05.525707 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.525706    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="22c541b8-e3ec-49b4-a002-1bad095243e4" containerName="manager"
Jul 28 20:00:05.528742 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.528721    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:05.531164 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.531129    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kbtxl\""
Jul 28 20:00:05.531311 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.531163    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 20:00:05.531311 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.531167    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 20:00:05.537049 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.537028    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-gn8nb"]
Jul 28 20:00:05.700624 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.700592    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3082458-3d5d-4e05-af90-e35520d421c7-webhook-cert\") pod \"maas-controller-5b5686dc45-gn8nb\" (UID: \"d3082458-3d5d-4e05-af90-e35520d421c7\") " pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:05.700756 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.700669    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjhc2\" (UniqueName: \"kubernetes.io/projected/d3082458-3d5d-4e05-af90-e35520d421c7-kube-api-access-qjhc2\") pod \"maas-controller-5b5686dc45-gn8nb\" (UID: \"d3082458-3d5d-4e05-af90-e35520d421c7\") " pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:05.801487 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.801416    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjhc2\" (UniqueName: \"kubernetes.io/projected/d3082458-3d5d-4e05-af90-e35520d421c7-kube-api-access-qjhc2\") pod \"maas-controller-5b5686dc45-gn8nb\" (UID: \"d3082458-3d5d-4e05-af90-e35520d421c7\") " pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:05.801600 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.801486    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3082458-3d5d-4e05-af90-e35520d421c7-webhook-cert\") pod \"maas-controller-5b5686dc45-gn8nb\" (UID: \"d3082458-3d5d-4e05-af90-e35520d421c7\") " pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:05.803925 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.803902    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3082458-3d5d-4e05-af90-e35520d421c7-webhook-cert\") pod \"maas-controller-5b5686dc45-gn8nb\" (UID: \"d3082458-3d5d-4e05-af90-e35520d421c7\") " pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:05.809469 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.809447    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjhc2\" (UniqueName: \"kubernetes.io/projected/d3082458-3d5d-4e05-af90-e35520d421c7-kube-api-access-qjhc2\") pod \"maas-controller-5b5686dc45-gn8nb\" (UID: \"d3082458-3d5d-4e05-af90-e35520d421c7\") " pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:05.840301 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.840269    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:05.969123 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.969090    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-gn8nb"]
Jul 28 20:00:05.971724 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:00:05.971698    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3082458_3d5d_4e05_af90_e35520d421c7.slice/crio-ab170aebea75e89a45cda6dcd8eae5787a55e52a8635f80146a8ccd49fdaad54 WatchSource:0}: Error finding container ab170aebea75e89a45cda6dcd8eae5787a55e52a8635f80146a8ccd49fdaad54: Status 404 returned error can't find the container with id ab170aebea75e89a45cda6dcd8eae5787a55e52a8635f80146a8ccd49fdaad54
Jul 28 20:00:05.973057 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:05.973037    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:00:06.008096 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:06.008073    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5686dc45-gn8nb" event={"ID":"d3082458-3d5d-4e05-af90-e35520d421c7","Type":"ContainerStarted","Data":"ab170aebea75e89a45cda6dcd8eae5787a55e52a8635f80146a8ccd49fdaad54"}
Jul 28 20:00:07.015650 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:07.015553    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5686dc45-gn8nb" event={"ID":"d3082458-3d5d-4e05-af90-e35520d421c7","Type":"ContainerStarted","Data":"3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c"}
Jul 28 20:00:07.015650 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:07.015626    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:07.033912 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:07.033867    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b5686dc45-gn8nb" podStartSLOduration=1.272637674 podStartE2EDuration="2.03385233s" totalImagesPullingTime="761.214656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:00:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:07.032048715 +0000 UTC m=+788.212565783" watchObservedRunningTime="2026-07-28 20:00:07.03385233 +0000 UTC m=+788.214369386"
Jul 28 20:00:18.023236 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:18.023204    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:22.558871 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:22.558790    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:00:22.662128 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:22.662101    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-gn8nb"]
Jul 28 20:00:22.662326 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:22.662304    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b5686dc45-gn8nb" podUID="d3082458-3d5d-4e05-af90-e35520d421c7" containerName="manager" containerID="cri-o://3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c" gracePeriod=30
Jul 28 20:00:22.909995 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:22.909964    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:22.941599 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:22.941570    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3082458-3d5d-4e05-af90-e35520d421c7-webhook-cert\") pod \"d3082458-3d5d-4e05-af90-e35520d421c7\" (UID: \"d3082458-3d5d-4e05-af90-e35520d421c7\") "
Jul 28 20:00:22.941763 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:22.941671    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjhc2\" (UniqueName: \"kubernetes.io/projected/d3082458-3d5d-4e05-af90-e35520d421c7-kube-api-access-qjhc2\") pod \"d3082458-3d5d-4e05-af90-e35520d421c7\" (UID: \"d3082458-3d5d-4e05-af90-e35520d421c7\") "
Jul 28 20:00:22.943791 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:22.943753    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3082458-3d5d-4e05-af90-e35520d421c7-kube-api-access-qjhc2" (OuterVolumeSpecName: "kube-api-access-qjhc2") pod "d3082458-3d5d-4e05-af90-e35520d421c7" (UID: "d3082458-3d5d-4e05-af90-e35520d421c7"). InnerVolumeSpecName "kube-api-access-qjhc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:00:22.944010 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:22.943989    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3082458-3d5d-4e05-af90-e35520d421c7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d3082458-3d5d-4e05-af90-e35520d421c7" (UID: "d3082458-3d5d-4e05-af90-e35520d421c7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:00:23.043094 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.043068    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3082458-3d5d-4e05-af90-e35520d421c7-webhook-cert\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:00:23.043094 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.043092    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjhc2\" (UniqueName: \"kubernetes.io/projected/d3082458-3d5d-4e05-af90-e35520d421c7-kube-api-access-qjhc2\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:00:23.092503 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.092443    2569 generic.go:356] "Generic (PLEG): container finished" podID="d3082458-3d5d-4e05-af90-e35520d421c7" containerID="3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c" exitCode=0
Jul 28 20:00:23.092503 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.092490    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5686dc45-gn8nb"
Jul 28 20:00:23.092626 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.092522    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5686dc45-gn8nb" event={"ID":"d3082458-3d5d-4e05-af90-e35520d421c7","Type":"ContainerDied","Data":"3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c"}
Jul 28 20:00:23.092626 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.092573    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5686dc45-gn8nb" event={"ID":"d3082458-3d5d-4e05-af90-e35520d421c7","Type":"ContainerDied","Data":"ab170aebea75e89a45cda6dcd8eae5787a55e52a8635f80146a8ccd49fdaad54"}
Jul 28 20:00:23.092626 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.092596    2569 scope.go:117] "RemoveContainer" containerID="3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c"
Jul 28 20:00:23.101645 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.101622    2569 scope.go:117] "RemoveContainer" containerID="3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c"
Jul 28 20:00:23.102266 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:00:23.101922    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c\": container with ID starting with 3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c not found: ID does not exist" containerID="3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c"
Jul 28 20:00:23.102266 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.101961    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c"} err="failed to get container status \"3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c\": rpc error: code = NotFound desc = could not find container \"3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c\": container with ID starting with 3c96b348454bb79ec61f937b08106db019f81731567a4634860281ce1982fe2c not found: ID does not exist"
Jul 28 20:00:23.114706 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.114681    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-gn8nb"]
Jul 28 20:00:23.117408 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.117387    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-gn8nb"]
Jul 28 20:00:23.366717 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.366651    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3082458-3d5d-4e05-af90-e35520d421c7" path="/var/lib/kubelet/pods/d3082458-3d5d-4e05-af90-e35520d421c7/volumes"
Jul 28 20:00:23.824845 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.824809    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:00:23.912976 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.912950    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b5686dc45-mjwfk"]
Jul 28 20:00:23.913418 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.913400    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3082458-3d5d-4e05-af90-e35520d421c7" containerName="manager"
Jul 28 20:00:23.913521 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.913420    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3082458-3d5d-4e05-af90-e35520d421c7" containerName="manager"
Jul 28 20:00:23.913575 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.913539    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3082458-3d5d-4e05-af90-e35520d421c7" containerName="manager"
Jul 28 20:00:23.918056 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.918037    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5686dc45-mjwfk"
Jul 28 20:00:23.920561 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.920541    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 20:00:23.920665 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.920605    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kbtxl\""
Jul 28 20:00:23.920665 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.920605    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 20:00:23.924553 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.924533    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-mjwfk"]
Jul 28 20:00:23.950236 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.950212    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90019d4c-c698-4d73-b8b9-f64c8f914318-webhook-cert\") pod \"maas-controller-5b5686dc45-mjwfk\" (UID: \"90019d4c-c698-4d73-b8b9-f64c8f914318\") " pod="opendatahub/maas-controller-5b5686dc45-mjwfk"
Jul 28 20:00:23.950345 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:23.950270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz4wk\" (UniqueName: \"kubernetes.io/projected/90019d4c-c698-4d73-b8b9-f64c8f914318-kube-api-access-bz4wk\") pod \"maas-controller-5b5686dc45-mjwfk\" (UID: \"90019d4c-c698-4d73-b8b9-f64c8f914318\") " pod="opendatahub/maas-controller-5b5686dc45-mjwfk"
Jul 28 20:00:24.051375 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:24.051352    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz4wk\" (UniqueName: \"kubernetes.io/projected/90019d4c-c698-4d73-b8b9-f64c8f914318-kube-api-access-bz4wk\") pod \"maas-controller-5b5686dc45-mjwfk\" (UID: \"90019d4c-c698-4d73-b8b9-f64c8f914318\") " pod="opendatahub/maas-controller-5b5686dc45-mjwfk"
Jul 28 20:00:24.051483 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:24.051415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90019d4c-c698-4d73-b8b9-f64c8f914318-webhook-cert\") pod \"maas-controller-5b5686dc45-mjwfk\" (UID: \"90019d4c-c698-4d73-b8b9-f64c8f914318\") " pod="opendatahub/maas-controller-5b5686dc45-mjwfk"
Jul 28 20:00:24.053491 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:24.053467    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90019d4c-c698-4d73-b8b9-f64c8f914318-webhook-cert\") pod \"maas-controller-5b5686dc45-mjwfk\" (UID: \"90019d4c-c698-4d73-b8b9-f64c8f914318\") " pod="opendatahub/maas-controller-5b5686dc45-mjwfk"
Jul 28 20:00:24.059653 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:24.059635    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz4wk\" (UniqueName: \"kubernetes.io/projected/90019d4c-c698-4d73-b8b9-f64c8f914318-kube-api-access-bz4wk\") pod \"maas-controller-5b5686dc45-mjwfk\" (UID: \"90019d4c-c698-4d73-b8b9-f64c8f914318\") " pod="opendatahub/maas-controller-5b5686dc45-mjwfk"
Jul 28 20:00:24.230918 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:24.230882    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b5686dc45-mjwfk"
Jul 28 20:00:24.366832 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:24.366808    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b5686dc45-mjwfk"]
Jul 28 20:00:24.369189 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:00:24.369164    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90019d4c_c698_4d73_b8b9_f64c8f914318.slice/crio-60a291864945994b939ab618c8527e2f2af4447c2290d10267f8be9d5430d0d1 WatchSource:0}: Error finding container 60a291864945994b939ab618c8527e2f2af4447c2290d10267f8be9d5430d0d1: Status 404 returned error can't find the container with id 60a291864945994b939ab618c8527e2f2af4447c2290d10267f8be9d5430d0d1
Jul 28 20:00:25.103919 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:25.103887    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5686dc45-mjwfk" event={"ID":"90019d4c-c698-4d73-b8b9-f64c8f914318","Type":"ContainerStarted","Data":"60a291864945994b939ab618c8527e2f2af4447c2290d10267f8be9d5430d0d1"}
Jul 28 20:00:26.109727 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:26.109683    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b5686dc45-mjwfk" event={"ID":"90019d4c-c698-4d73-b8b9-f64c8f914318","Type":"ContainerStarted","Data":"c3ceac2277a9a3a9617ea144af685a7c7d49a5c37796b1ebb88124e6aa6fb82f"}
Jul 28 20:00:26.110125 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:26.109841    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b5686dc45-mjwfk"
Jul 28 20:00:26.127317 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:26.127254    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b5686dc45-mjwfk" podStartSLOduration=2.162120785 podStartE2EDuration="3.127241181s" totalImagesPullingTime="965.120396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:26.125256635 +0000 UTC m=+807.305773690" watchObservedRunningTime="2026-07-28 20:00:26.127241181 +0000 UTC m=+807.307758242"
Jul 28 20:00:37.117329 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:00:37.117290    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b5686dc45-mjwfk"
Jul 28 20:01:01.979014 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:01.978976    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:02.080197 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:02.080161    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:02.182564 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:02.182530    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:07.983337 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:07.983304    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:08.089168 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:08.089131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:08.184167 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:08.184116    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:08.341476 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:08.341388    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:14.600061 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:14.600026    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:14.672795 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:14.672771    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:32.741677 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:32.741642    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:32.796801 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:32.796773    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:49.999629 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:49.999577    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:01:59.257993 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:59.257916    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:01:59.262545 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:01:59.262527    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:02:56.066329 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:02:56.066296    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:57.635335 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:03:57.635307    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:23.511792 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:23.511752    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:23.687799 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:23.687765    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:34.249037 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:34.249002    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"]
Jul 28 20:05:34.249557 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:34.249232    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8" podUID="03ff54f8-6c19-4448-86e7-6ef855f164e1" containerName="manager" containerID="cri-o://7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922" gracePeriod=10
Jul 28 20:05:37.504452 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.504426    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 20:05:37.593324 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.593234    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03ff54f8-6c19-4448-86e7-6ef855f164e1-extensions-socket-volume\") pod \"03ff54f8-6c19-4448-86e7-6ef855f164e1\" (UID: \"03ff54f8-6c19-4448-86e7-6ef855f164e1\") "
Jul 28 20:05:37.593324 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.593270    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m722q\" (UniqueName: \"kubernetes.io/projected/03ff54f8-6c19-4448-86e7-6ef855f164e1-kube-api-access-m722q\") pod \"03ff54f8-6c19-4448-86e7-6ef855f164e1\" (UID: \"03ff54f8-6c19-4448-86e7-6ef855f164e1\") "
Jul 28 20:05:37.593585 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.593560    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03ff54f8-6c19-4448-86e7-6ef855f164e1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "03ff54f8-6c19-4448-86e7-6ef855f164e1" (UID: "03ff54f8-6c19-4448-86e7-6ef855f164e1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:05:37.595323 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.595302    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03ff54f8-6c19-4448-86e7-6ef855f164e1-kube-api-access-m722q" (OuterVolumeSpecName: "kube-api-access-m722q") pod "03ff54f8-6c19-4448-86e7-6ef855f164e1" (UID: "03ff54f8-6c19-4448-86e7-6ef855f164e1"). InnerVolumeSpecName "kube-api-access-m722q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:05:37.608735 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.608709    2569 generic.go:356] "Generic (PLEG): container finished" podID="03ff54f8-6c19-4448-86e7-6ef855f164e1" containerID="7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922" exitCode=0
Jul 28 20:05:37.608839 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.608750    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"
Jul 28 20:05:37.608839 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.608771    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8" event={"ID":"03ff54f8-6c19-4448-86e7-6ef855f164e1","Type":"ContainerDied","Data":"7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922"}
Jul 28 20:05:37.608839 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.608814    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8" event={"ID":"03ff54f8-6c19-4448-86e7-6ef855f164e1","Type":"ContainerDied","Data":"656ca981fcc065785c97a292711d774ae1abdc56a42dc7871f5e674e536df1e9"}
Jul 28 20:05:37.608839 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.608835    2569 scope.go:117] "RemoveContainer" containerID="7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922"
Jul 28 20:05:37.625329 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.625300    2569 scope.go:117] "RemoveContainer" containerID="7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922"
Jul 28 20:05:37.625617 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:05:37.625597    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922\": container with ID starting with 7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922 not found: ID does not exist" containerID="7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922"
Jul 28 20:05:37.625685 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.625624    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922"} err="failed to get container status \"7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922\": rpc error: code = NotFound desc = could not find container \"7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922\": container with ID starting with 7b2d122433db521b62c95d4194ce27e52d81f5800ddcde904093e2594cd76922 not found: ID does not exist"
Jul 28 20:05:37.635210 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.635184    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"]
Jul 28 20:05:37.638569 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.638546    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kqkf8"]
Jul 28 20:05:37.694891 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.694858    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03ff54f8-6c19-4448-86e7-6ef855f164e1-extensions-socket-volume\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:05:37.694891 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:37.694891    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m722q\" (UniqueName: \"kubernetes.io/projected/03ff54f8-6c19-4448-86e7-6ef855f164e1-kube-api-access-m722q\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:05:39.366838 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:05:39.366798    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03ff54f8-6c19-4448-86e7-6ef855f164e1" path="/var/lib/kubelet/pods/03ff54f8-6c19-4448-86e7-6ef855f164e1/volumes"
Jul 28 20:06:35.483747 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.483674    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"]
Jul 28 20:06:35.484258 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.484056    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03ff54f8-6c19-4448-86e7-6ef855f164e1" containerName="manager"
Jul 28 20:06:35.484258 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.484069    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="03ff54f8-6c19-4448-86e7-6ef855f164e1" containerName="manager"
Jul 28 20:06:35.484258 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.484146    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="03ff54f8-6c19-4448-86e7-6ef855f164e1" containerName="manager"
Jul 28 20:06:35.487159 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.487143    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"
Jul 28 20:06:35.488993 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.488972    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nnmnz\""
Jul 28 20:06:35.489109 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.489007    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 20:06:35.489109 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.489032    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 20:06:35.496339 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.496317    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"]
Jul 28 20:06:35.559271 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.559247    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gnfr\" (UniqueName: \"kubernetes.io/projected/4384066d-b1a6-43f1-a184-cea1f130eb1c-kube-api-access-6gnfr\") pod \"kuadrant-operator-controller-manager-86878c769f-2g5sw\" (UID: \"4384066d-b1a6-43f1-a184-cea1f130eb1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"
Jul 28 20:06:35.559420 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.559364    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4384066d-b1a6-43f1-a184-cea1f130eb1c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2g5sw\" (UID: \"4384066d-b1a6-43f1-a184-cea1f130eb1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"
Jul 28 20:06:35.660083 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.660059    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4384066d-b1a6-43f1-a184-cea1f130eb1c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2g5sw\" (UID: \"4384066d-b1a6-43f1-a184-cea1f130eb1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"
Jul 28 20:06:35.660195 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.660119    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gnfr\" (UniqueName: \"kubernetes.io/projected/4384066d-b1a6-43f1-a184-cea1f130eb1c-kube-api-access-6gnfr\") pod \"kuadrant-operator-controller-manager-86878c769f-2g5sw\" (UID: \"4384066d-b1a6-43f1-a184-cea1f130eb1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"
Jul 28 20:06:35.660525 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.660506    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4384066d-b1a6-43f1-a184-cea1f130eb1c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2g5sw\" (UID: \"4384066d-b1a6-43f1-a184-cea1f130eb1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"
Jul 28 20:06:35.668332 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.668311    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gnfr\" (UniqueName: \"kubernetes.io/projected/4384066d-b1a6-43f1-a184-cea1f130eb1c-kube-api-access-6gnfr\") pod \"kuadrant-operator-controller-manager-86878c769f-2g5sw\" (UID: \"4384066d-b1a6-43f1-a184-cea1f130eb1c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"
Jul 28 20:06:35.798359 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.798268    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"
Jul 28 20:06:35.946501 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:35.944024    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"]
Jul 28 20:06:35.947061 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:06:35.947033    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4384066d_b1a6_43f1_a184_cea1f130eb1c.slice/crio-1793babdcb1c112effab17485fd297242715ea4afd1c130448603b3e163003db WatchSource:0}: Error finding container 1793babdcb1c112effab17485fd297242715ea4afd1c130448603b3e163003db: Status 404 returned error can't find the container with id 1793babdcb1c112effab17485fd297242715ea4afd1c130448603b3e163003db
Jul 28 20:06:36.887738 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:36.887702    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw" event={"ID":"4384066d-b1a6-43f1-a184-cea1f130eb1c","Type":"ContainerStarted","Data":"88ff7052f67cb3ee0e2f489331b11f30945d5b5017a3cfbfd973d0d080ecddbb"}
Jul 28 20:06:36.888217 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:36.887750    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw" event={"ID":"4384066d-b1a6-43f1-a184-cea1f130eb1c","Type":"ContainerStarted","Data":"1793babdcb1c112effab17485fd297242715ea4afd1c130448603b3e163003db"}
Jul 28 20:06:36.888217 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:36.887824    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"
Jul 28 20:06:36.907911 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:36.907861    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw" podStartSLOduration=1.9078446 podStartE2EDuration="1.9078446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:06:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:06:36.905805121 +0000 UTC m=+1178.086322177" watchObservedRunningTime="2026-07-28 20:06:36.9078446 +0000 UTC m=+1178.088361659"
Jul 28 20:06:47.894926 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:47.894893    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2g5sw"
Jul 28 20:06:59.288472 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:59.288444    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:06:59.293946 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:06:59.293926    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:07:24.554460 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:07:24.554422    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:08:45.500085 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:08:45.500050    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:20.376407 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.376367    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"]
Jul 28 20:09:20.380182 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.380165    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.382393 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.382361    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-4fcdad45-kserve-self-signed-certs\""
Jul 28 20:09:20.394023 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.394000    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"]
Jul 28 20:09:20.412803 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.412774    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:20.444942 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.444916    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xjwt\" (UniqueName: \"kubernetes.io/projected/fed3b2aa-ae4e-4614-ac11-460fd26caed5-kube-api-access-2xjwt\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.445238 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.445212    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-home\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.445337 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.445255    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tls-certs\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.445337 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.445288    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-dshm\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.445337 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.445313    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tmp-dir\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.445468 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.445388    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-model-cache\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.546572 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.546545    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xjwt\" (UniqueName: \"kubernetes.io/projected/fed3b2aa-ae4e-4614-ac11-460fd26caed5-kube-api-access-2xjwt\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.546705 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.546615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-home\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.546705 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.546640    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tls-certs\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.546705 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.546663    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-dshm\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.546705 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.546685    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tmp-dir\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.546911 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.546759    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-model-cache\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.547076 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.547055    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-home\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.547149 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.547119    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tmp-dir\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.547204 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.547185    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-model-cache\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.548918 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.548894    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-dshm\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.549154 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.549135    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tls-certs\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.553834 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.553816    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xjwt\" (UniqueName: \"kubernetes.io/projected/fed3b2aa-ae4e-4614-ac11-460fd26caed5-kube-api-access-2xjwt\") pod \"e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") " pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.691268 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.691239    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:20.820600 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:20.820575    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"]
Jul 28 20:09:20.822729 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:09:20.822699    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfed3b2aa_ae4e_4614_ac11_460fd26caed5.slice/crio-905c9c393ea8cce6b5d77f0ee171ae082b39d773bbf07d9cde102bd005c12e1d WatchSource:0}: Error finding container 905c9c393ea8cce6b5d77f0ee171ae082b39d773bbf07d9cde102bd005c12e1d: Status 404 returned error can't find the container with id 905c9c393ea8cce6b5d77f0ee171ae082b39d773bbf07d9cde102bd005c12e1d
Jul 28 20:09:21.666388 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:21.666312    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn" event={"ID":"fed3b2aa-ae4e-4614-ac11-460fd26caed5","Type":"ContainerStarted","Data":"ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17"}
Jul 28 20:09:21.666388 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:21.666353    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn" event={"ID":"fed3b2aa-ae4e-4614-ac11-460fd26caed5","Type":"ContainerStarted","Data":"905c9c393ea8cce6b5d77f0ee171ae082b39d773bbf07d9cde102bd005c12e1d"}
Jul 28 20:09:21.682713 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:21.682479    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn" podStartSLOduration=1.682438168 podStartE2EDuration="1.682438168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:21.682094002 +0000 UTC m=+1342.862611059" watchObservedRunningTime="2026-07-28 20:09:21.682438168 +0000 UTC m=+1342.862955223"
Jul 28 20:09:26.968567 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:26.968530    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"]
Jul 28 20:09:26.972319 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:26.972299    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:26.975447 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:26.975426    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-4fcdad45-kserve-self-signed-certs\""
Jul 28 20:09:26.982588 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:26.982569    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"]
Jul 28 20:09:26.992882 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:26.992862    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:27.108365 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.108331    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lctn\" (UniqueName: \"kubernetes.io/projected/b2a6cda3-78a6-49df-bd23-a6d08fce9594-kube-api-access-8lctn\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.108506 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.108374    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tls-certs\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.108506 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.108430    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-dshm\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.108506 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.108470    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-model-cache\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.108611 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.108506    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-home\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.108611 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.108576    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tmp-dir\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.209456 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.209428    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tls-certs\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.209593 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.209472    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-dshm\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.209593 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.209493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-model-cache\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.209593 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.209536    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-home\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.209733 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.209591    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tmp-dir\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.209733 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.209632    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lctn\" (UniqueName: \"kubernetes.io/projected/b2a6cda3-78a6-49df-bd23-a6d08fce9594-kube-api-access-8lctn\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.210040 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.210013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-home\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.210126 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.210021    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tmp-dir\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.210126 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.210072    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-model-cache\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.211880 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.211853    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-dshm\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.212273 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.212252    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tls-certs\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.217706 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.217685    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lctn\" (UniqueName: \"kubernetes.io/projected/b2a6cda3-78a6-49df-bd23-a6d08fce9594-kube-api-access-8lctn\") pod \"e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") " pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.282324 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.282256    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:27.423336 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.423306    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"]
Jul 28 20:09:27.425076 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:09:27.425050    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2a6cda3_78a6_49df_bd23_a6d08fce9594.slice/crio-001b9398b73bfb6439561c9f472843f761f29888c2a46886b82a28366ebfc138 WatchSource:0}: Error finding container 001b9398b73bfb6439561c9f472843f761f29888c2a46886b82a28366ebfc138: Status 404 returned error can't find the container with id 001b9398b73bfb6439561c9f472843f761f29888c2a46886b82a28366ebfc138
Jul 28 20:09:27.697754 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.697718    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6" event={"ID":"b2a6cda3-78a6-49df-bd23-a6d08fce9594","Type":"ContainerStarted","Data":"f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d"}
Jul 28 20:09:27.697959 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.697771    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6" event={"ID":"b2a6cda3-78a6-49df-bd23-a6d08fce9594","Type":"ContainerStarted","Data":"001b9398b73bfb6439561c9f472843f761f29888c2a46886b82a28366ebfc138"}
Jul 28 20:09:27.713460 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:27.713411    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6" podStartSLOduration=1.713390972 podStartE2EDuration="1.713390972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:09:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:09:27.71207458 +0000 UTC m=+1348.892591637" watchObservedRunningTime="2026-07-28 20:09:27.713390972 +0000 UTC m=+1348.893908034"
Jul 28 20:09:30.691821 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:30.691794    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:30.692263 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:30.691937    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:30.704376 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:30.704348    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:30.725985 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:30.725956    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:09:37.283021 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:37.282989    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:37.283388 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:37.283068    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:37.295204 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:37.295183    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:37.756820 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:37.756790    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:09:39.136886 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:39.136845    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:51.032023 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:51.031984    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"]
Jul 28 20:09:51.053108 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:51.053085    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:52.547089 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:52.547049    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"]
Jul 28 20:09:52.567110 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:09:52.567067    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:06.053444 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.053400    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn" podUID="fed3b2aa-ae4e-4614-ac11-460fd26caed5" containerName="main" containerID="cri-o://ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17" gracePeriod=45
Jul 28 20:10:06.299444 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.299422    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:10:06.360729 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.360607    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-home\") pod \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") "
Jul 28 20:10:06.361297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.360738    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tls-certs\") pod \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") "
Jul 28 20:10:06.361297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.360775    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xjwt\" (UniqueName: \"kubernetes.io/projected/fed3b2aa-ae4e-4614-ac11-460fd26caed5-kube-api-access-2xjwt\") pod \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") "
Jul 28 20:10:06.361297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.360830    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-model-cache\") pod \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") "
Jul 28 20:10:06.361297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.360858    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tmp-dir\") pod \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") "
Jul 28 20:10:06.361297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.360879    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-dshm\") pod \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\" (UID: \"fed3b2aa-ae4e-4614-ac11-460fd26caed5\") "
Jul 28 20:10:06.361297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.360774    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-home" (OuterVolumeSpecName: "home") pod "fed3b2aa-ae4e-4614-ac11-460fd26caed5" (UID: "fed3b2aa-ae4e-4614-ac11-460fd26caed5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:06.361297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.361084    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-model-cache" (OuterVolumeSpecName: "model-cache") pod "fed3b2aa-ae4e-4614-ac11-460fd26caed5" (UID: "fed3b2aa-ae4e-4614-ac11-460fd26caed5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:06.361297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.361166    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fed3b2aa-ae4e-4614-ac11-460fd26caed5" (UID: "fed3b2aa-ae4e-4614-ac11-460fd26caed5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:06.361297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.361187    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:06.361297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.361206    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:06.362761 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.362740    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-dshm" (OuterVolumeSpecName: "dshm") pod "fed3b2aa-ae4e-4614-ac11-460fd26caed5" (UID: "fed3b2aa-ae4e-4614-ac11-460fd26caed5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:06.362812 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.362801    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed3b2aa-ae4e-4614-ac11-460fd26caed5-kube-api-access-2xjwt" (OuterVolumeSpecName: "kube-api-access-2xjwt") pod "fed3b2aa-ae4e-4614-ac11-460fd26caed5" (UID: "fed3b2aa-ae4e-4614-ac11-460fd26caed5"). InnerVolumeSpecName "kube-api-access-2xjwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:10:06.362895 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.362877    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fed3b2aa-ae4e-4614-ac11-460fd26caed5" (UID: "fed3b2aa-ae4e-4614-ac11-460fd26caed5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:10:06.462485 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.462458    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:06.462485 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.462479    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xjwt\" (UniqueName: \"kubernetes.io/projected/fed3b2aa-ae4e-4614-ac11-460fd26caed5-kube-api-access-2xjwt\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:06.462485 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.462488    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:06.462743 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.462499    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fed3b2aa-ae4e-4614-ac11-460fd26caed5-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:06.891531 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.891497    2569 generic.go:356] "Generic (PLEG): container finished" podID="fed3b2aa-ae4e-4614-ac11-460fd26caed5" containerID="ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17" exitCode=0
Jul 28 20:10:06.891689 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.891541    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn" event={"ID":"fed3b2aa-ae4e-4614-ac11-460fd26caed5","Type":"ContainerDied","Data":"ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17"}
Jul 28 20:10:06.891689 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.891547    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"
Jul 28 20:10:06.891689 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.891587    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn" event={"ID":"fed3b2aa-ae4e-4614-ac11-460fd26caed5","Type":"ContainerDied","Data":"905c9c393ea8cce6b5d77f0ee171ae082b39d773bbf07d9cde102bd005c12e1d"}
Jul 28 20:10:06.891689 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.891613    2569 scope.go:117] "RemoveContainer" containerID="ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17"
Jul 28 20:10:06.901385 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.901366    2569 scope.go:117] "RemoveContainer" containerID="ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17"
Jul 28 20:10:06.901629 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:10:06.901613    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17\": container with ID starting with ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17 not found: ID does not exist" containerID="ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17"
Jul 28 20:10:06.901698 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.901635    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17"} err="failed to get container status \"ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17\": rpc error: code = NotFound desc = could not find container \"ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17\": container with ID starting with ad5ebee8a4831ec6998358aa1478692f34f996140fb53fb76b76a40923be6a17 not found: ID does not exist"
Jul 28 20:10:06.912166 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.912138    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"]
Jul 28 20:10:06.917800 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:06.917776    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-4fcdad45-kserve-55fc84c8c-88cvn"]
Jul 28 20:10:07.366293 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.366243    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fed3b2aa-ae4e-4614-ac11-460fd26caed5" path="/var/lib/kubelet/pods/fed3b2aa-ae4e-4614-ac11-460fd26caed5/volumes"
Jul 28 20:10:07.566360 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.566327    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6" podUID="b2a6cda3-78a6-49df-bd23-a6d08fce9594" containerName="main" containerID="cri-o://f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d" gracePeriod=45
Jul 28 20:10:07.820092 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.820067    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:10:07.876537 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.876516    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-home\") pod \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") "
Jul 28 20:10:07.876668 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.876549    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-dshm\") pod \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") "
Jul 28 20:10:07.876668 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.876581    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lctn\" (UniqueName: \"kubernetes.io/projected/b2a6cda3-78a6-49df-bd23-a6d08fce9594-kube-api-access-8lctn\") pod \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") "
Jul 28 20:10:07.876793 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.876666    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tmp-dir\") pod \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") "
Jul 28 20:10:07.876849 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.876790    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-model-cache\") pod \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") "
Jul 28 20:10:07.876849 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.876825    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-home" (OuterVolumeSpecName: "home") pod "b2a6cda3-78a6-49df-bd23-a6d08fce9594" (UID: "b2a6cda3-78a6-49df-bd23-a6d08fce9594"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:07.877040 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.876855    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tls-certs\") pod \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\" (UID: \"b2a6cda3-78a6-49df-bd23-a6d08fce9594\") "
Jul 28 20:10:07.877040 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.876861    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b2a6cda3-78a6-49df-bd23-a6d08fce9594" (UID: "b2a6cda3-78a6-49df-bd23-a6d08fce9594"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:07.877040 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.877001    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-model-cache" (OuterVolumeSpecName: "model-cache") pod "b2a6cda3-78a6-49df-bd23-a6d08fce9594" (UID: "b2a6cda3-78a6-49df-bd23-a6d08fce9594"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:07.877215 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.877196    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:07.877275 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.877219    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:07.877275 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.877233    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:07.878637 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.878617    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2a6cda3-78a6-49df-bd23-a6d08fce9594-kube-api-access-8lctn" (OuterVolumeSpecName: "kube-api-access-8lctn") pod "b2a6cda3-78a6-49df-bd23-a6d08fce9594" (UID: "b2a6cda3-78a6-49df-bd23-a6d08fce9594"). InnerVolumeSpecName "kube-api-access-8lctn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:10:07.878703 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.878651    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-dshm" (OuterVolumeSpecName: "dshm") pod "b2a6cda3-78a6-49df-bd23-a6d08fce9594" (UID: "b2a6cda3-78a6-49df-bd23-a6d08fce9594"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:10:07.878830 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.878813    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b2a6cda3-78a6-49df-bd23-a6d08fce9594" (UID: "b2a6cda3-78a6-49df-bd23-a6d08fce9594"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:10:07.898325 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.898299    2569 generic.go:356] "Generic (PLEG): container finished" podID="b2a6cda3-78a6-49df-bd23-a6d08fce9594" containerID="f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d" exitCode=0
Jul 28 20:10:07.898420 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.898335    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"
Jul 28 20:10:07.898420 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.898356    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6" event={"ID":"b2a6cda3-78a6-49df-bd23-a6d08fce9594","Type":"ContainerDied","Data":"f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d"}
Jul 28 20:10:07.898420 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.898384    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6" event={"ID":"b2a6cda3-78a6-49df-bd23-a6d08fce9594","Type":"ContainerDied","Data":"001b9398b73bfb6439561c9f472843f761f29888c2a46886b82a28366ebfc138"}
Jul 28 20:10:07.898420 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.898404    2569 scope.go:117] "RemoveContainer" containerID="f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d"
Jul 28 20:10:07.907397 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.907383    2569 scope.go:117] "RemoveContainer" containerID="f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d"
Jul 28 20:10:07.907635 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:10:07.907616    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d\": container with ID starting with f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d not found: ID does not exist" containerID="f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d"
Jul 28 20:10:07.907727 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.907640    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d"} err="failed to get container status \"f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d\": rpc error: code = NotFound desc = could not find container \"f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d\": container with ID starting with f113982ecd7d4cc8c64256f777e53287a73780f9ce8ee43fae2bb3a8918bb46d not found: ID does not exist"
Jul 28 20:10:07.919579 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.919550    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"]
Jul 28 20:10:07.923545 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.923521    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6"]
Jul 28 20:10:07.977940 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.977890    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2a6cda3-78a6-49df-bd23-a6d08fce9594-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:07.977940 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.977911    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2a6cda3-78a6-49df-bd23-a6d08fce9594-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:07.977940 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:07.977921    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lctn\" (UniqueName: \"kubernetes.io/projected/b2a6cda3-78a6-49df-bd23-a6d08fce9594-kube-api-access-8lctn\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:10:09.365435 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:09.365404    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2a6cda3-78a6-49df-bd23-a6d08fce9594" path="/var/lib/kubelet/pods/b2a6cda3-78a6-49df-bd23-a6d08fce9594/volumes"
Jul 28 20:10:12.747262 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:10:12.747218    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-b-4fcdad45-kserve-7f98dfcfff-9wkm6" podUID="b2a6cda3-78a6-49df-bd23-a6d08fce9594" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 28 20:11:59.319761 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:11:59.319726    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:11:59.326739 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:11:59.326715    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:13:07.137514 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:07.137480    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:29.364086 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:29.364052    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:32.505389 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:32.505351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:32.603779 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:32.603743    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:33.016436 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:33.016395    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:38.647808 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:38.647769    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:38.712120 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:38.712084    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:38.897944 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:38.897875    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:39.321067 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:39.321034    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:39.399699 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:39.399669    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:46.086260 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:46.086189    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:46.183247 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:46.183198    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:46.254974 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:46.254917    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:46.908583 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:46.908540    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:48.266941 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:13:48.266907    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:16.872854 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:16.872747    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:16.965785 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:16.965756    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:17.052096 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:17.052060    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:17.109896 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:17.109868    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:17.202316 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:17.202266    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:17.431177 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:17.431140    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:17.697252 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:17.697163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:17.786704 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:17.786669    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:17.893837 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:17.893806    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:18.255106 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:18.255070    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:19.071051 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:19.071019    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:20.891447 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:20.891411    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:20.983404 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:20.983372    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:21.079806 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:21.079761    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:21.425412 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:21.425378    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:21.529209 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:21.529178    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:51.506754 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:51.506716    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:51.572369 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:51.572337    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:51.637322 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:51.637248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:51.703914 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:51.703886    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:51.814114 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:51.814037    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:52.042785 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:52.042749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:52.281070 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:52.281039    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:52.609822 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:52.609745    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:53.118427 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:53.118391    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:55.375848 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:55.375807    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:55.475891 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:55.475853    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:55.551888 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:55.551853    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:55.806981 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:55.806949    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:56.156911 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:56.156838    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:56.252721 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:14:56.252682    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:00.131938 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.131906    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj"]
Jul 28 20:15:00.132318 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.132306    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed3b2aa-ae4e-4614-ac11-460fd26caed5" containerName="main"
Jul 28 20:15:00.132318 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.132320    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed3b2aa-ae4e-4614-ac11-460fd26caed5" containerName="main"
Jul 28 20:15:00.132400 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.132334    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2a6cda3-78a6-49df-bd23-a6d08fce9594" containerName="main"
Jul 28 20:15:00.132400 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.132340    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a6cda3-78a6-49df-bd23-a6d08fce9594" containerName="main"
Jul 28 20:15:00.132463 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.132401    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2a6cda3-78a6-49df-bd23-a6d08fce9594" containerName="main"
Jul 28 20:15:00.132463 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.132410    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed3b2aa-ae4e-4614-ac11-460fd26caed5" containerName="main"
Jul 28 20:15:00.135600 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.135581    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj"
Jul 28 20:15:00.137503 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.137483    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:15:00.143679 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.143609    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj"]
Jul 28 20:15:00.147215 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.147194    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj"
Jul 28 20:15:00.279521 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.279483    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj"]
Jul 28 20:15:00.281938 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:15:00.281843    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc9eaf3_af1e_45ef_8906_a23d3d767527.slice/crio-2edaefcc52f62b8ba3ddeb4c41cf9fe236bca39d8d47f09fdfd59e9c28230f76 WatchSource:0}: Error finding container 2edaefcc52f62b8ba3ddeb4c41cf9fe236bca39d8d47f09fdfd59e9c28230f76: Status 404 returned error can't find the container with id 2edaefcc52f62b8ba3ddeb4c41cf9fe236bca39d8d47f09fdfd59e9c28230f76
Jul 28 20:15:00.283802 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.283786    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:15:00.306927 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:00.306901    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj" event={"ID":"4bc9eaf3-af1e-45ef-8906-a23d3d767527","Type":"ContainerStarted","Data":"2edaefcc52f62b8ba3ddeb4c41cf9fe236bca39d8d47f09fdfd59e9c28230f76"}
Jul 28 20:15:03.323124 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:03.323090    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj" event={"ID":"4bc9eaf3-af1e-45ef-8906-a23d3d767527","Type":"ContainerStarted","Data":"12f6347f7c135e34e5f52b9e0dbf381fffd95d9780bf0656bf492d7aa8417abb"}
Jul 28 20:15:03.338084 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:03.338037    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj" podStartSLOduration=0.780501195 podStartE2EDuration="3.338021789s" totalImagesPullingTime="2.557520594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:03.336090968 +0000 UTC m=+1684.516608024" watchObservedRunningTime="2026-07-28 20:15:03.338021789 +0000 UTC m=+1684.518538845"
Jul 28 20:15:04.329513 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:04.329482    2569 generic.go:356] "Generic (PLEG): container finished" podID="4bc9eaf3-af1e-45ef-8906-a23d3d767527" containerID="12f6347f7c135e34e5f52b9e0dbf381fffd95d9780bf0656bf492d7aa8417abb" exitCode=0
Jul 28 20:15:04.329859 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:04.329555    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj" event={"ID":"4bc9eaf3-af1e-45ef-8906-a23d3d767527","Type":"ContainerDied","Data":"12f6347f7c135e34e5f52b9e0dbf381fffd95d9780bf0656bf492d7aa8417abb"}
Jul 28 20:15:05.468135 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:05.468113    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj"
Jul 28 20:15:06.341113 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:06.341083    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj"
Jul 28 20:15:06.341113 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:06.341101    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj" event={"ID":"4bc9eaf3-af1e-45ef-8906-a23d3d767527","Type":"ContainerDied","Data":"2edaefcc52f62b8ba3ddeb4c41cf9fe236bca39d8d47f09fdfd59e9c28230f76"}
Jul 28 20:15:06.341331 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:06.341147    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2edaefcc52f62b8ba3ddeb4c41cf9fe236bca39d8d47f09fdfd59e9c28230f76"
Jul 28 20:15:12.748436 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:12.748405    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:21.249759 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.249683    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"]
Jul 28 20:15:21.250210 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.250071    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bc9eaf3-af1e-45ef-8906-a23d3d767527" containerName="cleanup"
Jul 28 20:15:21.250210 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.250083    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bc9eaf3-af1e-45ef-8906-a23d3d767527" containerName="cleanup"
Jul 28 20:15:21.250210 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.250137    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bc9eaf3-af1e-45ef-8906-a23d3d767527" containerName="cleanup"
Jul 28 20:15:21.257658 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.257626    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"
Jul 28 20:15:21.259480 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.259453    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"]
Jul 28 20:15:21.259732 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.259711    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:15:21.259833 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.259711    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:15:21.349974 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.349943    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b2fb418-483f-47f4-aba1-8edd214d88b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq\" (UID: \"8b2fb418-483f-47f4-aba1-8edd214d88b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"
Jul 28 20:15:21.451341 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.451314    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b2fb418-483f-47f4-aba1-8edd214d88b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq\" (UID: \"8b2fb418-483f-47f4-aba1-8edd214d88b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"
Jul 28 20:15:21.451947 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.451928    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b2fb418-483f-47f4-aba1-8edd214d88b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq\" (UID: \"8b2fb418-483f-47f4-aba1-8edd214d88b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"
Jul 28 20:15:21.570007 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.569953    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"
Jul 28 20:15:21.703009 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:21.702985    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"]
Jul 28 20:15:21.705508 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:15:21.705481    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b2fb418_483f_47f4_aba1_8edd214d88b1.slice/crio-1ba6d2bff62dbc7d46c3d819a297d68fed3abbfbb89df7413126cc2c4c9efd34 WatchSource:0}: Error finding container 1ba6d2bff62dbc7d46c3d819a297d68fed3abbfbb89df7413126cc2c4c9efd34: Status 404 returned error can't find the container with id 1ba6d2bff62dbc7d46c3d819a297d68fed3abbfbb89df7413126cc2c4c9efd34
Jul 28 20:15:22.419339 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:22.419254    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq" event={"ID":"8b2fb418-483f-47f4-aba1-8edd214d88b1","Type":"ContainerStarted","Data":"17dbdaaafab00a822cfd66bed304e3efe5a1d6ee3ad666a491eb156dd3a89984"}
Jul 28 20:15:22.419339 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:22.419332    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq" event={"ID":"8b2fb418-483f-47f4-aba1-8edd214d88b1","Type":"ContainerStarted","Data":"1ba6d2bff62dbc7d46c3d819a297d68fed3abbfbb89df7413126cc2c4c9efd34"}
Jul 28 20:15:22.432674 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:22.432625    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq" podStartSLOduration=1.432609923 podStartE2EDuration="1.432609923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:22.431683241 +0000 UTC m=+1703.612200298" watchObservedRunningTime="2026-07-28 20:15:22.432609923 +0000 UTC m=+1703.613127032"
Jul 28 20:15:23.424844 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:23.424803    2569 generic.go:356] "Generic (PLEG): container finished" podID="8b2fb418-483f-47f4-aba1-8edd214d88b1" containerID="17dbdaaafab00a822cfd66bed304e3efe5a1d6ee3ad666a491eb156dd3a89984" exitCode=0
Jul 28 20:15:23.425312 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:23.424881    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq" event={"ID":"8b2fb418-483f-47f4-aba1-8edd214d88b1","Type":"ContainerDied","Data":"17dbdaaafab00a822cfd66bed304e3efe5a1d6ee3ad666a491eb156dd3a89984"}
Jul 28 20:15:24.582182 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:24.582158    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"
Jul 28 20:15:24.677903 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:24.677876    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b2fb418-483f-47f4-aba1-8edd214d88b1-maas-api-service-ca\") pod \"8b2fb418-483f-47f4-aba1-8edd214d88b1\" (UID: \"8b2fb418-483f-47f4-aba1-8edd214d88b1\") "
Jul 28 20:15:24.678192 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:24.678170    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b2fb418-483f-47f4-aba1-8edd214d88b1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8b2fb418-483f-47f4-aba1-8edd214d88b1" (UID: "8b2fb418-483f-47f4-aba1-8edd214d88b1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:15:24.778743 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:24.778672    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b2fb418-483f-47f4-aba1-8edd214d88b1-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:15:25.435790 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:25.435758    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq" event={"ID":"8b2fb418-483f-47f4-aba1-8edd214d88b1","Type":"ContainerDied","Data":"1ba6d2bff62dbc7d46c3d819a297d68fed3abbfbb89df7413126cc2c4c9efd34"}
Jul 28 20:15:25.435943 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:25.435790    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"
Jul 28 20:15:25.435943 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:25.435802    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ba6d2bff62dbc7d46c3d819a297d68fed3abbfbb89df7413126cc2c4c9efd34"
Jul 28 20:15:26.317162 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:26.317125    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj"]
Jul 28 20:15:26.320471 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:26.320444    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-edc4df58-29754495-srfgj"]
Jul 28 20:15:26.371150 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:26.371123    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:26.484472 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:26.484443    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:26.655289 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:26.655205    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"]
Jul 28 20:15:26.664681 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:26.664652    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-edc4df58-zlbpq"]
Jul 28 20:15:26.671047 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:26.671019    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:27.366010 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:27.365980    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bc9eaf3-af1e-45ef-8906-a23d3d767527" path="/var/lib/kubelet/pods/4bc9eaf3-af1e-45ef-8906-a23d3d767527/volumes"
Jul 28 20:15:27.366390 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:27.366267    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b2fb418-483f-47f4-aba1-8edd214d88b1" path="/var/lib/kubelet/pods/8b2fb418-483f-47f4-aba1-8edd214d88b1/volumes"
Jul 28 20:15:27.826324 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:27.826270    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:28.067562 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:28.067519    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:28.309546 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:28.309514    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:28.638888 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:28.638816    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:28.716390 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:28.716359    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:30.718430 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:30.718369    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:30.807041 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:30.807006    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:30.875875 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:30.875839    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:31.379828 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:31.379796    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:31.494091 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:31.494063    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:31.591734 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:31.591707    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:38.333730 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:38.333692    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:38.398715 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:38.398683    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:55.539583 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.539552    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"]
Jul 28 20:15:55.540034 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.539980    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b2fb418-483f-47f4-aba1-8edd214d88b1" containerName="revoke-keys"
Jul 28 20:15:55.540034 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.539992    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b2fb418-483f-47f4-aba1-8edd214d88b1" containerName="revoke-keys"
Jul 28 20:15:55.540107 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.540079    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b2fb418-483f-47f4-aba1-8edd214d88b1" containerName="revoke-keys"
Jul 28 20:15:55.543261 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.543245    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"
Jul 28 20:15:55.545533 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.545510    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:15:55.545701 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.545584    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:15:55.551196 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.551163    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"]
Jul 28 20:15:55.656458 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.656424    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f743894d-6170-4a03-b22f-136f2f0f4a4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6\" (UID: \"f743894d-6170-4a03-b22f-136f2f0f4a4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"
Jul 28 20:15:55.757880 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.757854    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f743894d-6170-4a03-b22f-136f2f0f4a4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6\" (UID: \"f743894d-6170-4a03-b22f-136f2f0f4a4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"
Jul 28 20:15:55.758491 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.758472    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f743894d-6170-4a03-b22f-136f2f0f4a4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6\" (UID: \"f743894d-6170-4a03-b22f-136f2f0f4a4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"
Jul 28 20:15:55.854604 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.854537    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"
Jul 28 20:15:55.980894 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:55.980868    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"]
Jul 28 20:15:55.984262 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:15:55.984231    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf743894d_6170_4a03_b22f_136f2f0f4a4b.slice/crio-712c81b6b042b0dce0bb136924f0397a5561c496981cc20f59bf2988b37a636a WatchSource:0}: Error finding container 712c81b6b042b0dce0bb136924f0397a5561c496981cc20f59bf2988b37a636a: Status 404 returned error can't find the container with id 712c81b6b042b0dce0bb136924f0397a5561c496981cc20f59bf2988b37a636a
Jul 28 20:15:56.580876 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:56.580801    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6" event={"ID":"f743894d-6170-4a03-b22f-136f2f0f4a4b","Type":"ContainerStarted","Data":"340399db2fb64169532d74dc8732246d85b6e8b21d7690afc400b27053e96889"}
Jul 28 20:15:56.581187 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:56.580878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6" event={"ID":"f743894d-6170-4a03-b22f-136f2f0f4a4b","Type":"ContainerStarted","Data":"712c81b6b042b0dce0bb136924f0397a5561c496981cc20f59bf2988b37a636a"}
Jul 28 20:15:56.595370 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:56.595319    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6" podStartSLOduration=1.595298976 podStartE2EDuration="1.595298976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:56.594563831 +0000 UTC m=+1737.775080888" watchObservedRunningTime="2026-07-28 20:15:56.595298976 +0000 UTC m=+1737.775816031"
Jul 28 20:15:57.586686 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:57.586652    2569 generic.go:356] "Generic (PLEG): container finished" podID="f743894d-6170-4a03-b22f-136f2f0f4a4b" containerID="340399db2fb64169532d74dc8732246d85b6e8b21d7690afc400b27053e96889" exitCode=0
Jul 28 20:15:57.587071 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:57.586739    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6" event={"ID":"f743894d-6170-4a03-b22f-136f2f0f4a4b","Type":"ContainerDied","Data":"340399db2fb64169532d74dc8732246d85b6e8b21d7690afc400b27053e96889"}
Jul 28 20:15:58.719152 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:58.719124    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"
Jul 28 20:15:58.780337 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:58.780299    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f743894d-6170-4a03-b22f-136f2f0f4a4b-maas-api-service-ca\") pod \"f743894d-6170-4a03-b22f-136f2f0f4a4b\" (UID: \"f743894d-6170-4a03-b22f-136f2f0f4a4b\") "
Jul 28 20:15:58.780601 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:58.780579    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f743894d-6170-4a03-b22f-136f2f0f4a4b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f743894d-6170-4a03-b22f-136f2f0f4a4b" (UID: "f743894d-6170-4a03-b22f-136f2f0f4a4b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:15:58.881312 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:58.881242    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f743894d-6170-4a03-b22f-136f2f0f4a4b-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:15:59.596907 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:59.596877    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"
Jul 28 20:15:59.597076 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:59.596922    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6" event={"ID":"f743894d-6170-4a03-b22f-136f2f0f4a4b","Type":"ContainerDied","Data":"712c81b6b042b0dce0bb136924f0397a5561c496981cc20f59bf2988b37a636a"}
Jul 28 20:15:59.597076 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:15:59.596967    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="712c81b6b042b0dce0bb136924f0397a5561c496981cc20f59bf2988b37a636a"
Jul 28 20:16:01.413867 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:01.413819    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:01.514487 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:01.514447    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:01.568836 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:01.568802    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"]
Jul 28 20:16:01.572964 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:01.572938    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-10cc1978-ldpr6"]
Jul 28 20:16:01.573510 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:01.573490    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:01.726978 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:01.726941    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:02.014807 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:02.014721    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:02.514984 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:02.514950    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:02.612646 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:02.612601    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:02.721180 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:02.721149    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:03.365429 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:03.365398    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f743894d-6170-4a03-b22f-136f2f0f4a4b" path="/var/lib/kubelet/pods/f743894d-6170-4a03-b22f-136f2f0f4a4b/volumes"
Jul 28 20:16:04.285675 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:04.285648    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:11.002168 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:11.002133    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:11.111667 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:11.111633    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:11.206920 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:11.206890    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:11.305546 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:11.305471    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:11.740939 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:11.740907    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:37.741258 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:37.741227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:43.995984 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:43.995950    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:44.066567 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:44.066538    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:44.145925 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:44.145893    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:44.245010 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:44.244984    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:59.354033 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:59.354006    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:16:59.361632 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:16:59.361609    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:17:00.099779 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:00.099746    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:05.710529 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:05.710495    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:05.813550 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:05.813511    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:05.911624 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:05.911598    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:16.476781 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:16.476749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:17.341681 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:17.341647    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:23.365615 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:23.365582    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:23.473092 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:23.473066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:23.566144 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:23.566114    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:23.666165 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:23.666088    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:35.195890 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:35.195853    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:35.294750 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:35.294719    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:35.395889 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:35.395862    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:37.202462 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:37.202427    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:43.111104 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:43.111066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:43.212359 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:43.212327    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:43.803646 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:43.803617    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:49.466489 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:49.466453    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:49.566423 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:49.566392    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:49.700214 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:49.700180    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:50.046495 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:50.046460    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:50.146892 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:50.146860    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:52.980985 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:52.980953    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:53.073271 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:53.073240    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:53.269793 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:53.269712    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:53.372575 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:53.372549    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:53.940075 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:53.940042    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:54.037423 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:17:54.037396    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:17.941127 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:17.941090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:18.628320 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:18.628271    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:18.733032 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:18.733004    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:19.087906 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:19.087871    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:24.766450 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:24.766374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:24.829130 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:24.829101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:25.151906 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.151837    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"]
Jul 28 20:18:25.152241 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.152228    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f743894d-6170-4a03-b22f-136f2f0f4a4b" containerName="revoke-keys"
Jul 28 20:18:25.152299 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.152242    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f743894d-6170-4a03-b22f-136f2f0f4a4b" containerName="revoke-keys"
Jul 28 20:18:25.152347 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.152326    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f743894d-6170-4a03-b22f-136f2f0f4a4b" containerName="revoke-keys"
Jul 28 20:18:25.155478 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.155459    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"
Jul 28 20:18:25.157587 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.157566    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:18:25.157675 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.157613    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:18:25.162122 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.162071    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"]
Jul 28 20:18:25.239353 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.239330    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ef67a0c-a010-404d-89fc-0139f53d64b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj\" (UID: \"1ef67a0c-a010-404d-89fc-0139f53d64b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"
Jul 28 20:18:25.340347 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.340326    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ef67a0c-a010-404d-89fc-0139f53d64b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj\" (UID: \"1ef67a0c-a010-404d-89fc-0139f53d64b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"
Jul 28 20:18:25.340954 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.340935    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ef67a0c-a010-404d-89fc-0139f53d64b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj\" (UID: \"1ef67a0c-a010-404d-89fc-0139f53d64b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"
Jul 28 20:18:25.467601 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.467579    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"
Jul 28 20:18:25.601127 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:25.601095    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"]
Jul 28 20:18:25.602418 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:18:25.602395    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef67a0c_a010_404d_89fc_0139f53d64b0.slice/crio-bb94e0c1d6874d4a605225527883a9e6ce86e7117e5c8b9c9672234faa07cc75 WatchSource:0}: Error finding container bb94e0c1d6874d4a605225527883a9e6ce86e7117e5c8b9c9672234faa07cc75: Status 404 returned error can't find the container with id bb94e0c1d6874d4a605225527883a9e6ce86e7117e5c8b9c9672234faa07cc75
Jul 28 20:18:26.332206 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:26.332076    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj" event={"ID":"1ef67a0c-a010-404d-89fc-0139f53d64b0","Type":"ContainerStarted","Data":"b95c8dc990a171f0a229cf7fe08a8a0ae30fb592da04cc1bf65e48d5faedb41e"}
Jul 28 20:18:26.332206 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:26.332138    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj" event={"ID":"1ef67a0c-a010-404d-89fc-0139f53d64b0","Type":"ContainerStarted","Data":"bb94e0c1d6874d4a605225527883a9e6ce86e7117e5c8b9c9672234faa07cc75"}
Jul 28 20:18:27.346246 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:27.346208    2569 generic.go:356] "Generic (PLEG): container finished" podID="1ef67a0c-a010-404d-89fc-0139f53d64b0" containerID="b95c8dc990a171f0a229cf7fe08a8a0ae30fb592da04cc1bf65e48d5faedb41e" exitCode=0
Jul 28 20:18:27.346672 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:27.346305    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj" event={"ID":"1ef67a0c-a010-404d-89fc-0139f53d64b0","Type":"ContainerDied","Data":"b95c8dc990a171f0a229cf7fe08a8a0ae30fb592da04cc1bf65e48d5faedb41e"}
Jul 28 20:18:27.479078 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:27.479054    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"
Jul 28 20:18:27.559084 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:27.559058    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ef67a0c-a010-404d-89fc-0139f53d64b0-maas-api-service-ca\") pod \"1ef67a0c-a010-404d-89fc-0139f53d64b0\" (UID: \"1ef67a0c-a010-404d-89fc-0139f53d64b0\") "
Jul 28 20:18:27.559431 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:27.559409    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ef67a0c-a010-404d-89fc-0139f53d64b0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1ef67a0c-a010-404d-89fc-0139f53d64b0" (UID: "1ef67a0c-a010-404d-89fc-0139f53d64b0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:18:27.660364 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:27.660298    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ef67a0c-a010-404d-89fc-0139f53d64b0-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:18:28.352495 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:28.352463    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj" event={"ID":"1ef67a0c-a010-404d-89fc-0139f53d64b0","Type":"ContainerDied","Data":"bb94e0c1d6874d4a605225527883a9e6ce86e7117e5c8b9c9672234faa07cc75"}
Jul 28 20:18:28.352838 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:28.352496    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"
Jul 28 20:18:28.352838 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:28.352512    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb94e0c1d6874d4a605225527883a9e6ce86e7117e5c8b9c9672234faa07cc75"
Jul 28 20:18:28.686275 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:28.686236    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:28.787078 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:28.787032    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:35.611378 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:35.611347    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:35.690488 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:35.690457    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:35.757751 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:35.757710    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:35.791076 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:35.791047    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"]
Jul 28 20:18:35.793820 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:35.793797    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6e5ce9cb-pm4sj"]
Jul 28 20:18:35.858419 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:35.858391    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:36.182368 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:36.182336    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:37.365907 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:37.365871    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ef67a0c-a010-404d-89fc-0139f53d64b0" path="/var/lib/kubelet/pods/1ef67a0c-a010-404d-89fc-0139f53d64b0/volumes"
Jul 28 20:18:41.979685 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:41.979645    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:42.037551 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:42.037508    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:42.241092 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:42.240988    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:42.484420 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:42.484386    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:42.612667 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:42.612593    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:42.710833 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:42.710804    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:42.841171 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:42.841136    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:48.548486 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:48.548447    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:49.759073 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:49.759035    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:54.707071 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:54.707037    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:54.774130 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:18:54.774099    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:01.753204 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:01.753168    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:02.057813 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:02.057741    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:02.260429 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:02.260400    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:05.348831 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:05.348792    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:05.448032 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:05.447999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:05.539488 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:05.539456    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:05.757449 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:05.757412    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:06.053497 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:06.053412    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:06.149766 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:06.149730    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:30.430083 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:30.430045    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:36.783882 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.783845    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"]
Jul 28 20:19:36.784495 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.784475    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef67a0c-a010-404d-89fc-0139f53d64b0" containerName="revoke-keys"
Jul 28 20:19:36.784555 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.784497    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef67a0c-a010-404d-89fc-0139f53d64b0" containerName="revoke-keys"
Jul 28 20:19:36.784624 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.784613    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef67a0c-a010-404d-89fc-0139f53d64b0" containerName="revoke-keys"
Jul 28 20:19:36.788006 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.787983    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:36.791862 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.791839    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3474a135\"/\"openshift-service-ca.crt\""
Jul 28 20:19:36.791979 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.791839    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3474a135\"/\"e2e-lifecycle-model-3474a135-kserve-self-signed-certs\""
Jul 28 20:19:36.791979 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.791844    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3474a135\"/\"kube-root-ca.crt\""
Jul 28 20:19:36.792242 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.792225    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3474a135\"/\"default-dockercfg-zqxx5\""
Jul 28 20:19:36.804895 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.804872    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"]
Jul 28 20:19:36.831553 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.831521    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:36.946583 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.946553    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-tmp-dir\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:36.946700 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.946610    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzpst\" (UniqueName: \"kubernetes.io/projected/22244c79-4db3-425d-aedc-c1273c0db4ed-kube-api-access-jzpst\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:36.946744 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.946698    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-model-cache\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:36.946783 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.946745    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22244c79-4db3-425d-aedc-c1273c0db4ed-tls-certs\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:36.946819 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.946787    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-home\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:36.946819 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:36.946808    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-dshm\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.047247 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.047189    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-home\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.047247 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.047220    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-dshm\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.047247 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.047243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-tmp-dir\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.047476 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.047273    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpst\" (UniqueName: \"kubernetes.io/projected/22244c79-4db3-425d-aedc-c1273c0db4ed-kube-api-access-jzpst\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.047476 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.047341    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-model-cache\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.047476 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.047379    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22244c79-4db3-425d-aedc-c1273c0db4ed-tls-certs\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.047677 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.047651    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-home\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.047780 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.047728    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-tmp-dir\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.047829 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.047810    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-model-cache\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.049450 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.049422    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-dshm\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.049776 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.049744    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22244c79-4db3-425d-aedc-c1273c0db4ed-tls-certs\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.055565 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.055545    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzpst\" (UniqueName: \"kubernetes.io/projected/22244c79-4db3-425d-aedc-c1273c0db4ed-kube-api-access-jzpst\") pod \"e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") " pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.099437 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.099416    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:37.230632 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.230597    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"]
Jul 28 20:19:37.233056 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:19:37.233028    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22244c79_4db3_425d_aedc_c1273c0db4ed.slice/crio-66b2a175eb5ac5d2a3f78a09256a1f0a90530d00d9c7cab6a2b2c9227cdbe9c7 WatchSource:0}: Error finding container 66b2a175eb5ac5d2a3f78a09256a1f0a90530d00d9c7cab6a2b2c9227cdbe9c7: Status 404 returned error can't find the container with id 66b2a175eb5ac5d2a3f78a09256a1f0a90530d00d9c7cab6a2b2c9227cdbe9c7
Jul 28 20:19:37.695957 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.695924    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j" event={"ID":"22244c79-4db3-425d-aedc-c1273c0db4ed","Type":"ContainerStarted","Data":"9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a"}
Jul 28 20:19:37.695957 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.695963    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j" event={"ID":"22244c79-4db3-425d-aedc-c1273c0db4ed","Type":"ContainerStarted","Data":"66b2a175eb5ac5d2a3f78a09256a1f0a90530d00d9c7cab6a2b2c9227cdbe9c7"}
Jul 28 20:19:37.713610 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:37.713559    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j" podStartSLOduration=1.713539433 podStartE2EDuration="1.713539433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:19:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:19:37.711520725 +0000 UTC m=+1958.892037781" watchObservedRunningTime="2026-07-28 20:19:37.713539433 +0000 UTC m=+1958.894056490"
Jul 28 20:19:47.099696 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:47.099662    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:47.099696 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:47.099708    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:47.111872 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:47.111841    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:19:47.758498 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:19:47.758470    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:20:15.653587 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:15.653534    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:20:15.750222 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:15.750191    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:20:15.912530 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:15.912435    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:20:16.019251 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:16.019222    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:20:19.357960 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:19.357926    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:20:24.412974 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:24.412937    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"]
Jul 28 20:20:24.512943 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:24.512906    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:20:35.666945 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:35.666907    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:20:39.432026 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.431983    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j" podUID="22244c79-4db3-425d-aedc-c1273c0db4ed" containerName="main" containerID="cri-o://9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a" gracePeriod=45
Jul 28 20:20:39.670402 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.670378    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:20:39.680705 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.680686    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22244c79-4db3-425d-aedc-c1273c0db4ed-tls-certs\") pod \"22244c79-4db3-425d-aedc-c1273c0db4ed\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") "
Jul 28 20:20:39.680824 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.680728    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-model-cache\") pod \"22244c79-4db3-425d-aedc-c1273c0db4ed\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") "
Jul 28 20:20:39.680889 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.680851    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-tmp-dir\") pod \"22244c79-4db3-425d-aedc-c1273c0db4ed\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") "
Jul 28 20:20:39.680948 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.680912    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-home\") pod \"22244c79-4db3-425d-aedc-c1273c0db4ed\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") "
Jul 28 20:20:39.681002 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.680946    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-dshm\") pod \"22244c79-4db3-425d-aedc-c1273c0db4ed\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") "
Jul 28 20:20:39.681002 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.680978    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "22244c79-4db3-425d-aedc-c1273c0db4ed" (UID: "22244c79-4db3-425d-aedc-c1273c0db4ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:20:39.681002 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.680999    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzpst\" (UniqueName: \"kubernetes.io/projected/22244c79-4db3-425d-aedc-c1273c0db4ed-kube-api-access-jzpst\") pod \"22244c79-4db3-425d-aedc-c1273c0db4ed\" (UID: \"22244c79-4db3-425d-aedc-c1273c0db4ed\") "
Jul 28 20:20:39.681165 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.681143    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "22244c79-4db3-425d-aedc-c1273c0db4ed" (UID: "22244c79-4db3-425d-aedc-c1273c0db4ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:20:39.681223 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.681170    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-home" (OuterVolumeSpecName: "home") pod "22244c79-4db3-425d-aedc-c1273c0db4ed" (UID: "22244c79-4db3-425d-aedc-c1273c0db4ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:20:39.681341 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.681320    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:20:39.681410 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.681343    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:20:39.681410 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.681352    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:20:39.682936 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.682887    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22244c79-4db3-425d-aedc-c1273c0db4ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "22244c79-4db3-425d-aedc-c1273c0db4ed" (UID: "22244c79-4db3-425d-aedc-c1273c0db4ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:20:39.683039 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.682931    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-dshm" (OuterVolumeSpecName: "dshm") pod "22244c79-4db3-425d-aedc-c1273c0db4ed" (UID: "22244c79-4db3-425d-aedc-c1273c0db4ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:20:39.683039 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.682935    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22244c79-4db3-425d-aedc-c1273c0db4ed-kube-api-access-jzpst" (OuterVolumeSpecName: "kube-api-access-jzpst") pod "22244c79-4db3-425d-aedc-c1273c0db4ed" (UID: "22244c79-4db3-425d-aedc-c1273c0db4ed"). InnerVolumeSpecName "kube-api-access-jzpst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:20:39.782490 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.782448    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzpst\" (UniqueName: \"kubernetes.io/projected/22244c79-4db3-425d-aedc-c1273c0db4ed-kube-api-access-jzpst\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:20:39.782490 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.782484    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22244c79-4db3-425d-aedc-c1273c0db4ed-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:20:39.782490 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:39.782495    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22244c79-4db3-425d-aedc-c1273c0db4ed-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:20:40.006387 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:40.006309    2569 generic.go:356] "Generic (PLEG): container finished" podID="22244c79-4db3-425d-aedc-c1273c0db4ed" containerID="9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a" exitCode=0
Jul 28 20:20:40.006387 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:40.006354    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"
Jul 28 20:20:40.006545 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:40.006390    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j" event={"ID":"22244c79-4db3-425d-aedc-c1273c0db4ed","Type":"ContainerDied","Data":"9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a"}
Jul 28 20:20:40.006545 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:40.006435    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j" event={"ID":"22244c79-4db3-425d-aedc-c1273c0db4ed","Type":"ContainerDied","Data":"66b2a175eb5ac5d2a3f78a09256a1f0a90530d00d9c7cab6a2b2c9227cdbe9c7"}
Jul 28 20:20:40.006545 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:40.006452    2569 scope.go:117] "RemoveContainer" containerID="9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a"
Jul 28 20:20:40.015423 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:40.015407    2569 scope.go:117] "RemoveContainer" containerID="9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a"
Jul 28 20:20:40.015656 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:20:40.015637    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a\": container with ID starting with 9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a not found: ID does not exist" containerID="9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a"
Jul 28 20:20:40.015719 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:40.015662    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a"} err="failed to get container status \"9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a\": rpc error: code = NotFound desc = could not find container \"9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a\": container with ID starting with 9c8af8c428a6d76a8598fb7eb9c9c7522c050c03d3c15982dc2d4bdf33b3787a not found: ID does not exist"
Jul 28 20:20:40.030647 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:40.030624    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"]
Jul 28 20:20:40.036792 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:40.036772    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3474a135/e2e-lifecycle-model-3474a135-kserve-7c549d947f-c5z4j"]
Jul 28 20:20:41.366099 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:41.366060    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22244c79-4db3-425d-aedc-c1273c0db4ed" path="/var/lib/kubelet/pods/22244c79-4db3-425d-aedc-c1273c0db4ed/volumes"
Jul 28 20:20:53.180064 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:20:53.180033    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:01.916900 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:01.916863    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:03.087196 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:03.087160    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:09.655660 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:09.655624    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:16.443122 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:16.443090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:16.641781 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:16.641751    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:16.860527 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:16.860452    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:17.145847 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:17.145767    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:18.479093 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:18.479057    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:18.611354 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:18.611325    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:19.560801 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:19.560767    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:19.703512 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:19.703482    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:21.305484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:21.305400    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:30.910503 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:30.910470    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:31.136982 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:31.136949    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:31.705971 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:31.705937    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:41.319325 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:41.319292    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:41.765271 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:41.765237    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:52.134645 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:52.134609    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:52.517361 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:52.517319    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:21:59.386616 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:59.386578    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:21:59.393746 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:59.393726    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:21:59.425609 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:59.425586    2569 scope.go:117] "RemoveContainer" containerID="17dbdaaafab00a822cfd66bed304e3efe5a1d6ee3ad666a491eb156dd3a89984"
Jul 28 20:21:59.433478 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:59.433461    2569 scope.go:117] "RemoveContainer" containerID="340399db2fb64169532d74dc8732246d85b6e8b21d7690afc400b27053e96889"
Jul 28 20:21:59.443418 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:21:59.443382    2569 scope.go:117] "RemoveContainer" containerID="12f6347f7c135e34e5f52b9e0dbf381fffd95d9780bf0656bf492d7aa8417abb"
Jul 28 20:22:13.707413 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:13.707378    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:23.937696 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:23.937663    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:24.270055 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:24.269978    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:24.633684 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:24.633596    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:25.278816 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:25.278784    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:25.486801 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:25.486773    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:29.039990 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:29.039953    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:29.289373 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:29.289318    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:29.670000 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:29.669969    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:30.039115 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:30.039066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:30.762065 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:30.762018    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:31.046930 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:31.046851    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:36.995735 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:36.995697    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:37.099273 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:37.099237    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:54.215615 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:54.215507    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:54.652195 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:54.652101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:55.254796 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:55.254769    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:55.611536 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:55.611445    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:58.936549 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:58.936515    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:59.144908 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:59.144876    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:59.348108 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:59.348023    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:59.663378 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:22:59.663306    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:23:00.559088 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:00.559058    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:23:00.928485 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:00.928455    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:23:09.967323 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:09.967267    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:23:23.809172 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:23.809138    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:23:34.773328 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.773297    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"]
Jul 28 20:23:34.773777 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.773681    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22244c79-4db3-425d-aedc-c1273c0db4ed" containerName="main"
Jul 28 20:23:34.773777 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.773692    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="22244c79-4db3-425d-aedc-c1273c0db4ed" containerName="main"
Jul 28 20:23:34.773777 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.773754    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="22244c79-4db3-425d-aedc-c1273c0db4ed" containerName="main"
Jul 28 20:23:34.776683 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.776664    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.779451 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.779434    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6dd19d\"/\"kube-root-ca.crt\""
Jul 28 20:23:34.780065 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.780051    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6dd19d\"/\"default-dockercfg-knnqf\""
Jul 28 20:23:34.780119 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.780052    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6dd19d\"/\"test-model-6dd19d-kserve-self-signed-certs\""
Jul 28 20:23:34.780156 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.780127    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6dd19d\"/\"openshift-service-ca.crt\""
Jul 28 20:23:34.791406 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.791386    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"]
Jul 28 20:23:34.806887 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.806864    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:23:34.854336 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.854309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-tmp-dir\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.854454 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.854346    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-dshm\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.854454 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.854426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-model-cache\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.854580 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.854467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2aaac068-801b-4799-92d0-570255a0550f-tls-certs\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.854580 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.854507    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-home\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.854652 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.854579    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9694z\" (UniqueName: \"kubernetes.io/projected/2aaac068-801b-4799-92d0-570255a0550f-kube-api-access-9694z\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.955746 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.955721    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-tmp-dir\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.955860 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.955750    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-dshm\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.955860 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.955776    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-model-cache\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.955860 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.955796    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2aaac068-801b-4799-92d0-570255a0550f-tls-certs\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.955860 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.955834    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-home\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.956116 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.955877    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9694z\" (UniqueName: \"kubernetes.io/projected/2aaac068-801b-4799-92d0-570255a0550f-kube-api-access-9694z\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.956177 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.956154    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-tmp-dir\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.956239 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.956161    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-model-cache\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.956329 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.956307    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-home\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.957990 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.957972    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-dshm\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.958148 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.958132    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2aaac068-801b-4799-92d0-570255a0550f-tls-certs\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:34.963347 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:34.963325    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9694z\" (UniqueName: \"kubernetes.io/projected/2aaac068-801b-4799-92d0-570255a0550f-kube-api-access-9694z\") pod \"test-model-6dd19d-kserve-5c8cf48957-klnrj\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") " pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:35.085652 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:35.085596    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:35.219924 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:35.219893    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"]
Jul 28 20:23:35.221908 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:23:35.221876    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2aaac068_801b_4799_92d0_570255a0550f.slice/crio-d438f8966124251d40a5a98a5ddf9e9eeb829b2f16da40fea62a2dad20a9653b WatchSource:0}: Error finding container d438f8966124251d40a5a98a5ddf9e9eeb829b2f16da40fea62a2dad20a9653b: Status 404 returned error can't find the container with id d438f8966124251d40a5a98a5ddf9e9eeb829b2f16da40fea62a2dad20a9653b
Jul 28 20:23:35.880131 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:35.880097    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj" event={"ID":"2aaac068-801b-4799-92d0-570255a0550f","Type":"ContainerStarted","Data":"980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e"}
Jul 28 20:23:35.880131 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:35.880137    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj" event={"ID":"2aaac068-801b-4799-92d0-570255a0550f","Type":"ContainerStarted","Data":"d438f8966124251d40a5a98a5ddf9e9eeb829b2f16da40fea62a2dad20a9653b"}
Jul 28 20:23:35.909423 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:35.909379    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj" podStartSLOduration=1.90936404 podStartE2EDuration="1.90936404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:23:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:23:35.908015692 +0000 UTC m=+2197.088532747" watchObservedRunningTime="2026-07-28 20:23:35.90936404 +0000 UTC m=+2197.089881096"
Jul 28 20:23:45.086533 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:45.086501    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:45.087017 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:45.086580    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:45.099055 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:45.099026    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:45.944197 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:45.944170    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:23:58.146776 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:23:58.146714    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:24:18.982891 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:18.982858    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"]
Jul 28 20:24:18.986648 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:18.986625    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:18.989616 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:18.989585    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-117350\"/\"openshift-service-ca.crt\""
Jul 28 20:24:18.989616 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:18.989611    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-117350\"/\"kube-root-ca.crt\""
Jul 28 20:24:18.989819 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:18.989640    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-117350\"/\"test-model-117350-kserve-self-signed-certs\""
Jul 28 20:24:18.989819 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:18.989652    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-117350\"/\"default-dockercfg-kgzp2\""
Jul 28 20:24:18.997433 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:18.997408    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"]
Jul 28 20:24:19.029068 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.029032    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e27969de-ec21-43dd-9ab1-dd509e71323a-tls-certs\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.029213 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.029088    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-home\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.029213 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.029142    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-model-cache\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.029360 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.029217    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-tmp-dir\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.029408 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.029365    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnk5z\" (UniqueName: \"kubernetes.io/projected/e27969de-ec21-43dd-9ab1-dd509e71323a-kube-api-access-rnk5z\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.029408 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.029388    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-dshm\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.029992 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.029975    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:24:19.137358 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.137319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnk5z\" (UniqueName: \"kubernetes.io/projected/e27969de-ec21-43dd-9ab1-dd509e71323a-kube-api-access-rnk5z\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.137500 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.137365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-dshm\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.137500 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.137428    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e27969de-ec21-43dd-9ab1-dd509e71323a-tls-certs\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.137596 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.137577    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-home\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.137635 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.137620    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-model-cache\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.137694 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.137657    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-tmp-dir\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.138028 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.137996    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-model-cache\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.138028 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.138019    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-home\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.138210 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.138074    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-tmp-dir\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.139795 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.139775    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-dshm\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.140017 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.139999    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e27969de-ec21-43dd-9ab1-dd509e71323a-tls-certs\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.146060 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.146038    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnk5z\" (UniqueName: \"kubernetes.io/projected/e27969de-ec21-43dd-9ab1-dd509e71323a-kube-api-access-rnk5z\") pod \"test-model-117350-kserve-57d576df55-96vwc\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") " pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.297134 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.297074    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:19.433889 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:19.433863    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"]
Jul 28 20:24:19.436502 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:24:19.436474    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode27969de_ec21_43dd_9ab1_dd509e71323a.slice/crio-bc26e95fe97263fe8ca4c2a94d500b7b41a52eaee95d09e3fdb8f7d31a4733ba WatchSource:0}: Error finding container bc26e95fe97263fe8ca4c2a94d500b7b41a52eaee95d09e3fdb8f7d31a4733ba: Status 404 returned error can't find the container with id bc26e95fe97263fe8ca4c2a94d500b7b41a52eaee95d09e3fdb8f7d31a4733ba
Jul 28 20:24:20.115223 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:20.115177    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc" event={"ID":"e27969de-ec21-43dd-9ab1-dd509e71323a","Type":"ContainerStarted","Data":"dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f"}
Jul 28 20:24:20.115682 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:20.115237    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc" event={"ID":"e27969de-ec21-43dd-9ab1-dd509e71323a","Type":"ContainerStarted","Data":"bc26e95fe97263fe8ca4c2a94d500b7b41a52eaee95d09e3fdb8f7d31a4733ba"}
Jul 28 20:24:20.137470 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:20.137421    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc" podStartSLOduration=2.137406096 podStartE2EDuration="2.137406096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:24:20.137322476 +0000 UTC m=+2241.317839533" watchObservedRunningTime="2026-07-28 20:24:20.137406096 +0000 UTC m=+2241.317923151"
Jul 28 20:24:29.297951 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:29.297869    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:29.297951 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:29.297916    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:29.310398 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:29.310368    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:30.176038 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:30.176006    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:24:59.480737 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:24:59.480706    2569 scope.go:117] "RemoveContainer" containerID="b95c8dc990a171f0a229cf7fe08a8a0ae30fb592da04cc1bf65e48d5faedb41e"
Jul 28 20:25:09.654972 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:09.654940    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"]
Jul 28 20:25:09.679165 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:09.679139    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:10.583944 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:10.583909    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"]
Jul 28 20:25:10.609041 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:10.609014    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:10.722469 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:10.722437    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"]
Jul 28 20:25:10.726517 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:10.726495    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"
Jul 28 20:25:10.731053 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:10.731028    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:25:10.731053 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:10.731047    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:25:10.734932 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:10.734912    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"]
Jul 28 20:25:10.786499 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:10.786478    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceddfab6-3eef-4770-8b95-18adc6d2f54d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx\" (UID: \"ceddfab6-3eef-4770-8b95-18adc6d2f54d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"
Jul 28 20:25:10.887901 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:10.887835    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceddfab6-3eef-4770-8b95-18adc6d2f54d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx\" (UID: \"ceddfab6-3eef-4770-8b95-18adc6d2f54d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"
Jul 28 20:25:10.888428 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:10.888411    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceddfab6-3eef-4770-8b95-18adc6d2f54d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx\" (UID: \"ceddfab6-3eef-4770-8b95-18adc6d2f54d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"
Jul 28 20:25:11.037513 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:11.037479    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"
Jul 28 20:25:11.170365 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:11.170342    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"]
Jul 28 20:25:11.172488 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:25:11.172458    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceddfab6_3eef_4770_8b95_18adc6d2f54d.slice/crio-bf62c8916be7b01d31a8d26f22a219f20fad65c353355fe5c9cb7c246b991002 WatchSource:0}: Error finding container bf62c8916be7b01d31a8d26f22a219f20fad65c353355fe5c9cb7c246b991002: Status 404 returned error can't find the container with id bf62c8916be7b01d31a8d26f22a219f20fad65c353355fe5c9cb7c246b991002
Jul 28 20:25:11.381678 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:11.381648    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx" event={"ID":"ceddfab6-3eef-4770-8b95-18adc6d2f54d","Type":"ContainerStarted","Data":"ffbccd50ad4695751db124737fa25d4db32cc637d2263d3ea3670ea152f65a2d"}
Jul 28 20:25:11.381774 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:11.381700    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx" event={"ID":"ceddfab6-3eef-4770-8b95-18adc6d2f54d","Type":"ContainerStarted","Data":"bf62c8916be7b01d31a8d26f22a219f20fad65c353355fe5c9cb7c246b991002"}
Jul 28 20:25:11.404567 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:11.404526    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx" podStartSLOduration=1.40451065 podStartE2EDuration="1.40451065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:11.402332309 +0000 UTC m=+2292.582849365" watchObservedRunningTime="2026-07-28 20:25:11.40451065 +0000 UTC m=+2292.585027748"
Jul 28 20:25:12.388217 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:12.388182    2569 generic.go:356] "Generic (PLEG): container finished" podID="ceddfab6-3eef-4770-8b95-18adc6d2f54d" containerID="ffbccd50ad4695751db124737fa25d4db32cc637d2263d3ea3670ea152f65a2d" exitCode=0
Jul 28 20:25:12.388625 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:12.388252    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx" event={"ID":"ceddfab6-3eef-4770-8b95-18adc6d2f54d","Type":"ContainerDied","Data":"ffbccd50ad4695751db124737fa25d4db32cc637d2263d3ea3670ea152f65a2d"}
Jul 28 20:25:13.527924 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:13.527894    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"
Jul 28 20:25:13.613674 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:13.613641    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceddfab6-3eef-4770-8b95-18adc6d2f54d-maas-api-service-ca\") pod \"ceddfab6-3eef-4770-8b95-18adc6d2f54d\" (UID: \"ceddfab6-3eef-4770-8b95-18adc6d2f54d\") "
Jul 28 20:25:13.613937 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:13.613915    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceddfab6-3eef-4770-8b95-18adc6d2f54d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ceddfab6-3eef-4770-8b95-18adc6d2f54d" (UID: "ceddfab6-3eef-4770-8b95-18adc6d2f54d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:25:13.715124 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:13.715097    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceddfab6-3eef-4770-8b95-18adc6d2f54d-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:14.399548 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:14.399522    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"
Jul 28 20:25:14.399717 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:14.399574    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx" event={"ID":"ceddfab6-3eef-4770-8b95-18adc6d2f54d","Type":"ContainerDied","Data":"bf62c8916be7b01d31a8d26f22a219f20fad65c353355fe5c9cb7c246b991002"}
Jul 28 20:25:14.399717 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:14.399611    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf62c8916be7b01d31a8d26f22a219f20fad65c353355fe5c9cb7c246b991002"
Jul 28 20:25:14.745946 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:14.745916    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:14.828188 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:14.828157    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:14.932103 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:14.932042    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:14.965894 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:14.965853    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"]
Jul 28 20:25:14.967956 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:14.967923    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-6dd19d-9dqlx"]
Jul 28 20:25:15.030388 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:15.030327    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:15.365697 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:15.365622    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceddfab6-3eef-4770-8b95-18adc6d2f54d" path="/var/lib/kubelet/pods/ceddfab6-3eef-4770-8b95-18adc6d2f54d/volumes"
Jul 28 20:25:15.385212 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:15.385181    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:20.440502 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:20.440469    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:24.673453 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:24.673413    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj" podUID="2aaac068-801b-4799-92d0-570255a0550f" containerName="main" containerID="cri-o://980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e" gracePeriod=45
Jul 28 20:25:24.930702 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:24.930647    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:25:25.016956 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.016931    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2aaac068-801b-4799-92d0-570255a0550f-tls-certs\") pod \"2aaac068-801b-4799-92d0-570255a0550f\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") "
Jul 28 20:25:25.017098 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.016979    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-tmp-dir\") pod \"2aaac068-801b-4799-92d0-570255a0550f\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") "
Jul 28 20:25:25.017098 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.016997    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-home\") pod \"2aaac068-801b-4799-92d0-570255a0550f\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") "
Jul 28 20:25:25.017232 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.017120    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9694z\" (UniqueName: \"kubernetes.io/projected/2aaac068-801b-4799-92d0-570255a0550f-kube-api-access-9694z\") pod \"2aaac068-801b-4799-92d0-570255a0550f\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") "
Jul 28 20:25:25.017232 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.017179    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-model-cache\") pod \"2aaac068-801b-4799-92d0-570255a0550f\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") "
Jul 28 20:25:25.017232 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.017207    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-home" (OuterVolumeSpecName: "home") pod "2aaac068-801b-4799-92d0-570255a0550f" (UID: "2aaac068-801b-4799-92d0-570255a0550f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:25:25.017443 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.017225    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2aaac068-801b-4799-92d0-570255a0550f" (UID: "2aaac068-801b-4799-92d0-570255a0550f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:25:25.017443 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.017320    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-dshm\") pod \"2aaac068-801b-4799-92d0-570255a0550f\" (UID: \"2aaac068-801b-4799-92d0-570255a0550f\") "
Jul 28 20:25:25.017548 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.017481    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-model-cache" (OuterVolumeSpecName: "model-cache") pod "2aaac068-801b-4799-92d0-570255a0550f" (UID: "2aaac068-801b-4799-92d0-570255a0550f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:25:25.017682 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.017662    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:25.017742 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.017686    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:25.017742 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.017697    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:25.019239 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.019216    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2aaac068-801b-4799-92d0-570255a0550f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2aaac068-801b-4799-92d0-570255a0550f" (UID: "2aaac068-801b-4799-92d0-570255a0550f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:25:25.019356 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.019309    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aaac068-801b-4799-92d0-570255a0550f-kube-api-access-9694z" (OuterVolumeSpecName: "kube-api-access-9694z") pod "2aaac068-801b-4799-92d0-570255a0550f" (UID: "2aaac068-801b-4799-92d0-570255a0550f"). InnerVolumeSpecName "kube-api-access-9694z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:25:25.019541 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.019519    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-dshm" (OuterVolumeSpecName: "dshm") pod "2aaac068-801b-4799-92d0-570255a0550f" (UID: "2aaac068-801b-4799-92d0-570255a0550f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:25:25.118792 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.118766    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2aaac068-801b-4799-92d0-570255a0550f-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:25.118792 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.118787    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9694z\" (UniqueName: \"kubernetes.io/projected/2aaac068-801b-4799-92d0-570255a0550f-kube-api-access-9694z\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:25.118792 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.118796    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2aaac068-801b-4799-92d0-570255a0550f-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:25.453178 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.453148    2569 generic.go:356] "Generic (PLEG): container finished" podID="2aaac068-801b-4799-92d0-570255a0550f" containerID="980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e" exitCode=0
Jul 28 20:25:25.453338 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.453186    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"
Jul 28 20:25:25.453338 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.453232    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj" event={"ID":"2aaac068-801b-4799-92d0-570255a0550f","Type":"ContainerDied","Data":"980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e"}
Jul 28 20:25:25.453338 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.453273    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj" event={"ID":"2aaac068-801b-4799-92d0-570255a0550f","Type":"ContainerDied","Data":"d438f8966124251d40a5a98a5ddf9e9eeb829b2f16da40fea62a2dad20a9653b"}
Jul 28 20:25:25.453338 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.453302    2569 scope.go:117] "RemoveContainer" containerID="980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e"
Jul 28 20:25:25.461720 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.461704    2569 scope.go:117] "RemoveContainer" containerID="980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e"
Jul 28 20:25:25.461959 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:25:25.461937    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e\": container with ID starting with 980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e not found: ID does not exist" containerID="980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e"
Jul 28 20:25:25.462057 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.461963    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e"} err="failed to get container status \"980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e\": rpc error: code = NotFound desc = could not find container \"980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e\": container with ID starting with 980a1c711669c2e7ee186e25a249951fd35614e6df48711942b6fa4490ae107e not found: ID does not exist"
Jul 28 20:25:25.462150 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.462125    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:25.470892 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.470870    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"]
Jul 28 20:25:25.474358 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.474337    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6dd19d/test-model-6dd19d-kserve-5c8cf48957-klnrj"]
Jul 28 20:25:25.602524 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.602498    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc" podUID="e27969de-ec21-43dd-9ab1-dd509e71323a" containerName="main" containerID="cri-o://dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f" gracePeriod=45
Jul 28 20:25:25.854987 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:25.854959    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:25:26.028885 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.028815    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-model-cache\") pod \"e27969de-ec21-43dd-9ab1-dd509e71323a\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") "
Jul 28 20:25:26.028885 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.028870    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-dshm\") pod \"e27969de-ec21-43dd-9ab1-dd509e71323a\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") "
Jul 28 20:25:26.029056 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.028911    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e27969de-ec21-43dd-9ab1-dd509e71323a-tls-certs\") pod \"e27969de-ec21-43dd-9ab1-dd509e71323a\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") "
Jul 28 20:25:26.029056 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.028936    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-home\") pod \"e27969de-ec21-43dd-9ab1-dd509e71323a\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") "
Jul 28 20:25:26.029056 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.028958    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-tmp-dir\") pod \"e27969de-ec21-43dd-9ab1-dd509e71323a\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") "
Jul 28 20:25:26.029056 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.029008    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnk5z\" (UniqueName: \"kubernetes.io/projected/e27969de-ec21-43dd-9ab1-dd509e71323a-kube-api-access-rnk5z\") pod \"e27969de-ec21-43dd-9ab1-dd509e71323a\" (UID: \"e27969de-ec21-43dd-9ab1-dd509e71323a\") "
Jul 28 20:25:26.029252 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.029074    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-model-cache" (OuterVolumeSpecName: "model-cache") pod "e27969de-ec21-43dd-9ab1-dd509e71323a" (UID: "e27969de-ec21-43dd-9ab1-dd509e71323a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:25:26.029252 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.029209    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e27969de-ec21-43dd-9ab1-dd509e71323a" (UID: "e27969de-ec21-43dd-9ab1-dd509e71323a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:25:26.029252 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.029226    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-home" (OuterVolumeSpecName: "home") pod "e27969de-ec21-43dd-9ab1-dd509e71323a" (UID: "e27969de-ec21-43dd-9ab1-dd509e71323a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:25:26.029443 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.029383    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:26.029443 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.029419    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:26.029443 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.029440    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:26.031163 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.031138    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e27969de-ec21-43dd-9ab1-dd509e71323a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e27969de-ec21-43dd-9ab1-dd509e71323a" (UID: "e27969de-ec21-43dd-9ab1-dd509e71323a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:25:26.031295 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.031179    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-dshm" (OuterVolumeSpecName: "dshm") pod "e27969de-ec21-43dd-9ab1-dd509e71323a" (UID: "e27969de-ec21-43dd-9ab1-dd509e71323a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:25:26.031295 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.031268    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e27969de-ec21-43dd-9ab1-dd509e71323a-kube-api-access-rnk5z" (OuterVolumeSpecName: "kube-api-access-rnk5z") pod "e27969de-ec21-43dd-9ab1-dd509e71323a" (UID: "e27969de-ec21-43dd-9ab1-dd509e71323a"). InnerVolumeSpecName "kube-api-access-rnk5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:25:26.130764 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.130731    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnk5z\" (UniqueName: \"kubernetes.io/projected/e27969de-ec21-43dd-9ab1-dd509e71323a-kube-api-access-rnk5z\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:26.130764 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.130758    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e27969de-ec21-43dd-9ab1-dd509e71323a-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:26.130764 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.130768    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e27969de-ec21-43dd-9ab1-dd509e71323a-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:26.459318 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.459268    2569 generic.go:356] "Generic (PLEG): container finished" podID="e27969de-ec21-43dd-9ab1-dd509e71323a" containerID="dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f" exitCode=0
Jul 28 20:25:26.459318 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.459304    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc" event={"ID":"e27969de-ec21-43dd-9ab1-dd509e71323a","Type":"ContainerDied","Data":"dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f"}
Jul 28 20:25:26.459555 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.459355    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc" event={"ID":"e27969de-ec21-43dd-9ab1-dd509e71323a","Type":"ContainerDied","Data":"bc26e95fe97263fe8ca4c2a94d500b7b41a52eaee95d09e3fdb8f7d31a4733ba"}
Jul 28 20:25:26.459555 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.459320    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"
Jul 28 20:25:26.459555 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.459378    2569 scope.go:117] "RemoveContainer" containerID="dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f"
Jul 28 20:25:26.468847 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.468831    2569 scope.go:117] "RemoveContainer" containerID="dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f"
Jul 28 20:25:26.469083 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:25:26.469063    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f\": container with ID starting with dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f not found: ID does not exist" containerID="dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f"
Jul 28 20:25:26.469147 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.469095    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f"} err="failed to get container status \"dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f\": rpc error: code = NotFound desc = could not find container \"dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f\": container with ID starting with dbd2ab83d80732094038a2b3c4e4c0b408391ef0342b1c32a6634afee47d0b8f not found: ID does not exist"
Jul 28 20:25:26.486677 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.486655    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"]
Jul 28 20:25:26.491710 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:26.491689    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-117350/test-model-117350-kserve-57d576df55-96vwc"]
Jul 28 20:25:27.292681 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.292641    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:27.365977 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.365944    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2aaac068-801b-4799-92d0-570255a0550f" path="/var/lib/kubelet/pods/2aaac068-801b-4799-92d0-570255a0550f/volumes"
Jul 28 20:25:27.366413 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.366396    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e27969de-ec21-43dd-9ab1-dd509e71323a" path="/var/lib/kubelet/pods/e27969de-ec21-43dd-9ab1-dd509e71323a/volumes"
Jul 28 20:25:27.394086 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.394064    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:27.621023 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.620944    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:27.865608 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.865579    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:27.969694 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.969657    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"]
Jul 28 20:25:27.970072 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.970056    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceddfab6-3eef-4770-8b95-18adc6d2f54d" containerName="revoke-keys"
Jul 28 20:25:27.970152 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.970074    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceddfab6-3eef-4770-8b95-18adc6d2f54d" containerName="revoke-keys"
Jul 28 20:25:27.970152 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.970101    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aaac068-801b-4799-92d0-570255a0550f" containerName="main"
Jul 28 20:25:27.970152 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.970108    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aaac068-801b-4799-92d0-570255a0550f" containerName="main"
Jul 28 20:25:27.970152 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.970114    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e27969de-ec21-43dd-9ab1-dd509e71323a" containerName="main"
Jul 28 20:25:27.970152 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.970121    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e27969de-ec21-43dd-9ab1-dd509e71323a" containerName="main"
Jul 28 20:25:27.970446 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.970187    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2aaac068-801b-4799-92d0-570255a0550f" containerName="main"
Jul 28 20:25:27.970446 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.970198    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e27969de-ec21-43dd-9ab1-dd509e71323a" containerName="main"
Jul 28 20:25:27.970446 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.970206    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceddfab6-3eef-4770-8b95-18adc6d2f54d" containerName="revoke-keys"
Jul 28 20:25:27.974700 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.974679    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"
Jul 28 20:25:27.977559 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.977047    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:25:27.977559 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.977307    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:25:27.980669 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:27.980647    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"]
Jul 28 20:25:28.145167 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:28.145140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e81133-b153-4aa0-84e5-2a9085e0c80c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-117350-7pzw9\" (UID: \"29e81133-b153-4aa0-84e5-2a9085e0c80c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"
Jul 28 20:25:28.245941 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:28.245877    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e81133-b153-4aa0-84e5-2a9085e0c80c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-117350-7pzw9\" (UID: \"29e81133-b153-4aa0-84e5-2a9085e0c80c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"
Jul 28 20:25:28.246492 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:28.246475    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e81133-b153-4aa0-84e5-2a9085e0c80c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-117350-7pzw9\" (UID: \"29e81133-b153-4aa0-84e5-2a9085e0c80c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"
Jul 28 20:25:28.287217 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:28.287189    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"
Jul 28 20:25:28.428094 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:28.428071    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"]
Jul 28 20:25:28.430437 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:25:28.430409    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e81133_b153_4aa0_84e5_2a9085e0c80c.slice/crio-18cb990300b6cf009f65263aa4e959819f10b198d5ca96196cefa71c654bdaa1 WatchSource:0}: Error finding container 18cb990300b6cf009f65263aa4e959819f10b198d5ca96196cefa71c654bdaa1: Status 404 returned error can't find the container with id 18cb990300b6cf009f65263aa4e959819f10b198d5ca96196cefa71c654bdaa1
Jul 28 20:25:28.473230 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:28.473204    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9" event={"ID":"29e81133-b153-4aa0-84e5-2a9085e0c80c","Type":"ContainerStarted","Data":"18cb990300b6cf009f65263aa4e959819f10b198d5ca96196cefa71c654bdaa1"}
Jul 28 20:25:29.479384 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:29.479352    2569 generic.go:356] "Generic (PLEG): container finished" podID="29e81133-b153-4aa0-84e5-2a9085e0c80c" containerID="b1af6c83337c576f613c26e64ff1b7fc0bb3d8bcd4959415a612d9489b697de1" exitCode=0
Jul 28 20:25:29.479773 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:29.479439    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9" event={"ID":"29e81133-b153-4aa0-84e5-2a9085e0c80c","Type":"ContainerDied","Data":"b1af6c83337c576f613c26e64ff1b7fc0bb3d8bcd4959415a612d9489b697de1"}
Jul 28 20:25:30.613424 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:30.613399    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"
Jul 28 20:25:30.770098 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:30.770011    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e81133-b153-4aa0-84e5-2a9085e0c80c-maas-api-service-ca\") pod \"29e81133-b153-4aa0-84e5-2a9085e0c80c\" (UID: \"29e81133-b153-4aa0-84e5-2a9085e0c80c\") "
Jul 28 20:25:30.770393 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:30.770370    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29e81133-b153-4aa0-84e5-2a9085e0c80c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29e81133-b153-4aa0-84e5-2a9085e0c80c" (UID: "29e81133-b153-4aa0-84e5-2a9085e0c80c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:25:30.870966 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:30.870932    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e81133-b153-4aa0-84e5-2a9085e0c80c-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:25:31.489938 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:31.489913    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"
Jul 28 20:25:31.489938 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:31.489923    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9" event={"ID":"29e81133-b153-4aa0-84e5-2a9085e0c80c","Type":"ContainerDied","Data":"18cb990300b6cf009f65263aa4e959819f10b198d5ca96196cefa71c654bdaa1"}
Jul 28 20:25:31.490115 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:31.489964    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18cb990300b6cf009f65263aa4e959819f10b198d5ca96196cefa71c654bdaa1"
Jul 28 20:25:38.514721 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:38.514676    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:38.685631 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:38.685599    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:38.789823 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:38.789748    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:38.826242 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:38.826212    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"]
Jul 28 20:25:38.830945 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:38.830920    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-117350-7pzw9"]
Jul 28 20:25:38.864644 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:38.864618    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:39.243245 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:39.243215    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:39.366093 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:39.366062    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29e81133-b153-4aa0-84e5-2a9085e0c80c" path="/var/lib/kubelet/pods/29e81133-b153-4aa0-84e5-2a9085e0c80c/volumes"
Jul 28 20:25:42.955417 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:42.955388    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:45.481949 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:45.481913    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:45.556623 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:45.556595    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:45.807007 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:45.806919    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:46.367766 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:46.367734    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:46.924779 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:46.924727    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:47.390339 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:47.390245    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:47.578659 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:47.578627    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:51.225817 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:51.225724    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:51.362885 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:51.362857    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:51.501490 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:51.501416    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:51.652606 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:51.652575    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:52.221773 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:52.221738    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:52.333439 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:25:52.333405    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:16.298960 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:16.298929    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.123150 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:17.123116    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.480245 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:17.480215    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.600248 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:17.600215    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:21.049177 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:21.049138    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:21.254889 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:21.254848    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:21.353338 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:21.353249    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:21.465742 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:21.465703    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:21.858922 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:21.858891    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:21.961157 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:21.961122    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:51.191604 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:51.191570    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:54.474369 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.474334    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"]
Jul 28 20:26:54.474805 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.474746    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29e81133-b153-4aa0-84e5-2a9085e0c80c" containerName="revoke-keys"
Jul 28 20:26:54.474805 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.474758    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="29e81133-b153-4aa0-84e5-2a9085e0c80c" containerName="revoke-keys"
Jul 28 20:26:54.474878 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.474816    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="29e81133-b153-4aa0-84e5-2a9085e0c80c" containerName="revoke-keys"
Jul 28 20:26:54.477862 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.477841    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"
Jul 28 20:26:54.483025 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.483001    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:26:54.483143 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.483033    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:26:54.503952 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.503927    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"]
Jul 28 20:26:54.587507 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.587478    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e4badf8-59cd-42c2-b065-7ca244236b22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-711431-2247x\" (UID: \"9e4badf8-59cd-42c2-b065-7ca244236b22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"
Jul 28 20:26:54.688127 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.688090    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e4badf8-59cd-42c2-b065-7ca244236b22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-711431-2247x\" (UID: \"9e4badf8-59cd-42c2-b065-7ca244236b22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"
Jul 28 20:26:54.688678 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.688660    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e4badf8-59cd-42c2-b065-7ca244236b22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-711431-2247x\" (UID: \"9e4badf8-59cd-42c2-b065-7ca244236b22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"
Jul 28 20:26:54.787405 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.787331    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"
Jul 28 20:26:54.938674 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:54.938649    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"]
Jul 28 20:26:54.941416 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:26:54.941391    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e4badf8_59cd_42c2_b065_7ca244236b22.slice/crio-5b598786594ae5b8eae0588abf185a16212d446d0b97b7d5cad8174415f2830b WatchSource:0}: Error finding container 5b598786594ae5b8eae0588abf185a16212d446d0b97b7d5cad8174415f2830b: Status 404 returned error can't find the container with id 5b598786594ae5b8eae0588abf185a16212d446d0b97b7d5cad8174415f2830b
Jul 28 20:26:55.913870 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:55.913842    2569 generic.go:356] "Generic (PLEG): container finished" podID="9e4badf8-59cd-42c2-b065-7ca244236b22" containerID="96f9ead6925fc8f632465728a555e0c78e032a0f1171f2d38bb45f951ae0256c" exitCode=0
Jul 28 20:26:55.914165 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:55.913912    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x" event={"ID":"9e4badf8-59cd-42c2-b065-7ca244236b22","Type":"ContainerDied","Data":"96f9ead6925fc8f632465728a555e0c78e032a0f1171f2d38bb45f951ae0256c"}
Jul 28 20:26:55.914165 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:55.913957    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x" event={"ID":"9e4badf8-59cd-42c2-b065-7ca244236b22","Type":"ContainerStarted","Data":"5b598786594ae5b8eae0588abf185a16212d446d0b97b7d5cad8174415f2830b"}
Jul 28 20:26:57.051474 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:57.051449    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"
Jul 28 20:26:57.209306 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:57.209269    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e4badf8-59cd-42c2-b065-7ca244236b22-maas-api-service-ca\") pod \"9e4badf8-59cd-42c2-b065-7ca244236b22\" (UID: \"9e4badf8-59cd-42c2-b065-7ca244236b22\") "
Jul 28 20:26:57.209671 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:57.209647    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e4badf8-59cd-42c2-b065-7ca244236b22-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e4badf8-59cd-42c2-b065-7ca244236b22" (UID: "9e4badf8-59cd-42c2-b065-7ca244236b22"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:26:57.310747 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:57.310716    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e4badf8-59cd-42c2-b065-7ca244236b22-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:26:57.931631 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:57.931600    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"
Jul 28 20:26:57.931822 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:57.931637    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x" event={"ID":"9e4badf8-59cd-42c2-b065-7ca244236b22","Type":"ContainerDied","Data":"5b598786594ae5b8eae0588abf185a16212d446d0b97b7d5cad8174415f2830b"}
Jul 28 20:26:57.931822 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:57.931703    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b598786594ae5b8eae0588abf185a16212d446d0b97b7d5cad8174415f2830b"
Jul 28 20:26:59.417856 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:59.417829    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:26:59.428313 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:26:59.428267    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:27:10.120077 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:10.120040    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:10.218813 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:10.218773    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:10.301677 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:10.301644    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:10.343513 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:10.343483    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"]
Jul 28 20:27:10.349468 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:10.349443    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-711431-2247x"]
Jul 28 20:27:10.376624 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:10.376566    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:10.765734 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:10.765711    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:11.365969 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:11.365927    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e4badf8-59cd-42c2-b065-7ca244236b22" path="/var/lib/kubelet/pods/9e4badf8-59cd-42c2-b065-7ca244236b22/volumes"
Jul 28 20:27:17.499499 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:17.499462    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:17.562637 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:17.562608    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:17.811910 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:17.811819    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:18.054612 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.054567    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:18.168186 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.168097    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"]
Jul 28 20:27:18.168495 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.168480    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e4badf8-59cd-42c2-b065-7ca244236b22" containerName="revoke-keys"
Jul 28 20:27:18.168495 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.168495    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e4badf8-59cd-42c2-b065-7ca244236b22" containerName="revoke-keys"
Jul 28 20:27:18.168655 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.168573    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e4badf8-59cd-42c2-b065-7ca244236b22" containerName="revoke-keys"
Jul 28 20:27:18.171751 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.171727    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"
Jul 28 20:27:18.174018 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.173994    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:27:18.174018 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.174007    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:27:18.176336 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.176316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/274cb837-7d71-4499-9361-b27a0b697373-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2\" (UID: \"274cb837-7d71-4499-9361-b27a0b697373\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"
Jul 28 20:27:18.187956 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.187929    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"]
Jul 28 20:27:18.277674 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.277648    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/274cb837-7d71-4499-9361-b27a0b697373-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2\" (UID: \"274cb837-7d71-4499-9361-b27a0b697373\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"
Jul 28 20:27:18.278267 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.278246    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/274cb837-7d71-4499-9361-b27a0b697373-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2\" (UID: \"274cb837-7d71-4499-9361-b27a0b697373\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"
Jul 28 20:27:18.481909 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.481884    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"
Jul 28 20:27:18.639113 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:18.639085    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"]
Jul 28 20:27:18.641474 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:27:18.641446    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod274cb837_7d71_4499_9361_b27a0b697373.slice/crio-104d79914d28a7f7d2bfc1d9febf697b3cc8d770b1a2165e4603aa9702bd2148 WatchSource:0}: Error finding container 104d79914d28a7f7d2bfc1d9febf697b3cc8d770b1a2165e4603aa9702bd2148: Status 404 returned error can't find the container with id 104d79914d28a7f7d2bfc1d9febf697b3cc8d770b1a2165e4603aa9702bd2148
Jul 28 20:27:19.038814 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:19.038782    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2" event={"ID":"274cb837-7d71-4499-9361-b27a0b697373","Type":"ContainerStarted","Data":"144a31862283e1a3db1f3e59463c2060576ecdfc22b091d823081e7572aed78c"}
Jul 28 20:27:19.038988 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:19.038869    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2" event={"ID":"274cb837-7d71-4499-9361-b27a0b697373","Type":"ContainerStarted","Data":"104d79914d28a7f7d2bfc1d9febf697b3cc8d770b1a2165e4603aa9702bd2148"}
Jul 28 20:27:19.055111 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:19.055066    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2" podStartSLOduration=1.055050632 podStartE2EDuration="1.055050632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:27:19.053983827 +0000 UTC m=+2420.234500902" watchObservedRunningTime="2026-07-28 20:27:19.055050632 +0000 UTC m=+2420.235567728"
Jul 28 20:27:20.045529 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:20.045495    2569 generic.go:356] "Generic (PLEG): container finished" podID="274cb837-7d71-4499-9361-b27a0b697373" containerID="144a31862283e1a3db1f3e59463c2060576ecdfc22b091d823081e7572aed78c" exitCode=0
Jul 28 20:27:20.045946 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:20.045582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2" event={"ID":"274cb837-7d71-4499-9361-b27a0b697373","Type":"ContainerDied","Data":"144a31862283e1a3db1f3e59463c2060576ecdfc22b091d823081e7572aed78c"}
Jul 28 20:27:21.181265 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:21.181237    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"
Jul 28 20:27:21.206206 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:21.206183    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/274cb837-7d71-4499-9361-b27a0b697373-maas-api-service-ca\") pod \"274cb837-7d71-4499-9361-b27a0b697373\" (UID: \"274cb837-7d71-4499-9361-b27a0b697373\") "
Jul 28 20:27:21.206564 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:21.206541    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/274cb837-7d71-4499-9361-b27a0b697373-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "274cb837-7d71-4499-9361-b27a0b697373" (UID: "274cb837-7d71-4499-9361-b27a0b697373"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:27:21.307706 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:21.307619    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/274cb837-7d71-4499-9361-b27a0b697373-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:27:22.057472 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:22.057444    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"
Jul 28 20:27:22.057629 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:22.057444    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2" event={"ID":"274cb837-7d71-4499-9361-b27a0b697373","Type":"ContainerDied","Data":"104d79914d28a7f7d2bfc1d9febf697b3cc8d770b1a2165e4603aa9702bd2148"}
Jul 28 20:27:22.057629 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:22.057565    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="104d79914d28a7f7d2bfc1d9febf697b3cc8d770b1a2165e4603aa9702bd2148"
Jul 28 20:27:24.418249 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:24.418217    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:33.740876 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:33.740846    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:33.823318 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:33.823257    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:33.897560 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:33.897529    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:33.958428 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:33.958397    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"]
Jul 28 20:27:33.964042 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:33.964018    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f632f1-lhjh2"]
Jul 28 20:27:34.003796 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:34.003740    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:34.383027 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:34.382941    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:35.366025 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:35.365998    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="274cb837-7d71-4499-9361-b27a0b697373" path="/var/lib/kubelet/pods/274cb837-7d71-4499-9361-b27a0b697373/volumes"
Jul 28 20:27:40.983041 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:40.983003    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:41.084292 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:41.084257    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:41.262042 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:41.261966    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:41.587842 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:41.587767    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:41.918631 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:41.918555    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:42.353229 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:42.353201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:42.697419 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:42.697390    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:45.383442 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:45.383406    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:45.484317 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:45.484247    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:45.682562 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:45.682537    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:45.819826 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:45.819797    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:46.458537 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:46.458505    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:46.640724 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:27:46.640692    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:10.385950 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:10.385918    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:10.743535 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:10.743498    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:11.114900 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:11.114823    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:11.487165 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:11.487136    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:14.692130 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:14.692101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:14.848977 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:14.848942    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:14.972273 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:14.972192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:15.321758 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:15.321671    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:15.970248 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:15.970218    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:16.143353 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:16.143320    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:39.656725 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:39.656695    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:39.751589 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:39.751562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:58.100138 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.100106    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"]
Jul 28 20:28:58.100819 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.100526    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="274cb837-7d71-4499-9361-b27a0b697373" containerName="revoke-keys"
Jul 28 20:28:58.100819 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.100538    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="274cb837-7d71-4499-9361-b27a0b697373" containerName="revoke-keys"
Jul 28 20:28:58.100819 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.100605    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="274cb837-7d71-4499-9361-b27a0b697373" containerName="revoke-keys"
Jul 28 20:28:58.103805 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.103788    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.106028 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.106002    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ed54db\"/\"default-dockercfg-l7hfk\""
Jul 28 20:28:58.106492 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.106471    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ed54db\"/\"kube-root-ca.crt\""
Jul 28 20:28:58.106580 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.106558    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ed54db\"/\"openshift-service-ca.crt\""
Jul 28 20:28:58.106634 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.106598    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ed54db\"/\"auth-test-model-ed54db-kserve-self-signed-certs\""
Jul 28 20:28:58.121319 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.121272    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"]
Jul 28 20:28:58.139056 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.139031    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tls-certs\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.139150 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.139082    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-model-cache\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.139150 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.139099    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tmp-dir\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.139235 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.139161    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q49sw\" (UniqueName: \"kubernetes.io/projected/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-kube-api-access-q49sw\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.139235 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.139184    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-home\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.139334 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.139253    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-dshm\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.158235 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.158154    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:58.239739 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.239709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-model-cache\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.239739 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.239739    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tmp-dir\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.239887 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.239867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q49sw\" (UniqueName: \"kubernetes.io/projected/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-kube-api-access-q49sw\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.239928 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.239903    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-home\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.239978 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.239926    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-dshm\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.240035 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.239985    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tls-certs\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.240123 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.240102    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tmp-dir\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.240182 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.240118    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-model-cache\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.240267 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.240238    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-home\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.242084 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.242054    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-dshm\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.242235 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.242215    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tls-certs\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.249131 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.249106    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q49sw\" (UniqueName: \"kubernetes.io/projected/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-kube-api-access-q49sw\") pod \"auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") " pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.413412 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.413342    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:28:58.555843 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:58.555815    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"]
Jul 28 20:28:59.546345 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:59.546308    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm" event={"ID":"7e800be2-bfcc-454c-9f36-e4dabfbf54c3","Type":"ContainerStarted","Data":"20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212"}
Jul 28 20:28:59.546704 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:59.546355    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm" event={"ID":"7e800be2-bfcc-454c-9f36-e4dabfbf54c3","Type":"ContainerStarted","Data":"b81a8475db9f7d1db038fb1e7ced90e718525afe2b08fd272015addcf77a5f3c"}
Jul 28 20:28:59.567081 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:28:59.567037    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm" podStartSLOduration=1.56702187 podStartE2EDuration="1.56702187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:28:59.564470965 +0000 UTC m=+2520.744988021" watchObservedRunningTime="2026-07-28 20:28:59.56702187 +0000 UTC m=+2520.747538926"
Jul 28 20:29:08.414130 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:08.414093    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:29:08.414130 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:08.414144    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:29:08.426489 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:08.426463    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:29:08.600862 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:08.600838    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:29:18.902092 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:18.902059    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"]
Jul 28 20:29:18.905515 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:18.905495    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:18.911068 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:18.909820    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"default-dockercfg-l6d6k\""
Jul 28 20:29:18.911431 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:18.911402    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"kube-root-ca.crt\""
Jul 28 20:29:18.912175 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:18.911555    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"auth-test-model-1231ba-kserve-self-signed-certs\""
Jul 28 20:29:18.912175 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:18.912076    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"openshift-service-ca.crt\""
Jul 28 20:29:18.927122 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:18.927098    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"]
Jul 28 20:29:19.022110 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.022058    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:19.028351 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.028320    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-home\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.028485 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.028406    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-model-cache\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.028485 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.028428    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-tmp-dir\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.028485 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.028468    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmnz8\" (UniqueName: \"kubernetes.io/projected/cd5e2766-2447-488c-9441-7b5647be4527-kube-api-access-zmnz8\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.028637 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.028533    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-dshm\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.028637 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.028622    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2766-2447-488c-9441-7b5647be4527-tls-certs\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.130033 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.130004    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2766-2447-488c-9441-7b5647be4527-tls-certs\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.130191 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.130046    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-home\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.130191 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.130109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-model-cache\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.130191 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.130140    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-tmp-dir\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.130384 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.130198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmnz8\" (UniqueName: \"kubernetes.io/projected/cd5e2766-2447-488c-9441-7b5647be4527-kube-api-access-zmnz8\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.130384 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.130240    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-dshm\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.130483 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.130463    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-home\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.130527 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.130509    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-model-cache\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.130584 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.130560    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-tmp-dir\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.132362 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.132336    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-dshm\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.132513 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.132494    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2766-2447-488c-9441-7b5647be4527-tls-certs\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.140013 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.139990    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmnz8\" (UniqueName: \"kubernetes.io/projected/cd5e2766-2447-488c-9441-7b5647be4527-kube-api-access-zmnz8\") pod \"auth-test-model-1231ba-kserve-59bf4696c5-rnczj\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") " pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.217247 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.217224    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:19.357836 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.357811    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"]
Jul 28 20:29:19.359852 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:29:19.359807    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd5e2766_2447_488c_9441_7b5647be4527.slice/crio-803c4e6e5a36eb24fdb7f05dda83ad0131034845a05ec6abbde0ff49a344f7e8 WatchSource:0}: Error finding container 803c4e6e5a36eb24fdb7f05dda83ad0131034845a05ec6abbde0ff49a344f7e8: Status 404 returned error can't find the container with id 803c4e6e5a36eb24fdb7f05dda83ad0131034845a05ec6abbde0ff49a344f7e8
Jul 28 20:29:19.652637 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.652550    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj" event={"ID":"cd5e2766-2447-488c-9441-7b5647be4527","Type":"ContainerStarted","Data":"2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490"}
Jul 28 20:29:19.652637 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.652596    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj" event={"ID":"cd5e2766-2447-488c-9441-7b5647be4527","Type":"ContainerStarted","Data":"803c4e6e5a36eb24fdb7f05dda83ad0131034845a05ec6abbde0ff49a344f7e8"}
Jul 28 20:29:19.672889 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:19.672844    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj" podStartSLOduration=1.672829502 podStartE2EDuration="1.672829502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:29:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:29:19.671125772 +0000 UTC m=+2540.851642828" watchObservedRunningTime="2026-07-28 20:29:19.672829502 +0000 UTC m=+2540.853346558"
Jul 28 20:29:29.218253 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:29.218216    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:29.218723 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:29.218272    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:29.230519 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:29.230495    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:29:29.718118 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:29:29.718087    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:30:00.140250 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.140218    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l"]
Jul 28 20:30:00.144265 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.144241    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc"]
Jul 28 20:30:00.144431 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.144356    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l"
Jul 28 20:30:00.146479 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.146458    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:30:00.147717 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.147701    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8"]
Jul 28 20:30:00.147860 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.147838    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc"
Jul 28 20:30:00.151229 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.151199    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8"
Jul 28 20:30:00.153345 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.153319    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l"]
Jul 28 20:30:00.156536 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.156519    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l"
Jul 28 20:30:00.164795 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.161911    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8"]
Jul 28 20:30:00.164795 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.164221    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc"]
Jul 28 20:30:00.169060 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.169041    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc"
Jul 28 20:30:00.206686 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.206504    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8"
Jul 28 20:30:00.334189 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.334144    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l"]
Jul 28 20:30:00.338180 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:30:00.338155    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f451faa_6e75_48b4_be4a_cefe3327b9bb.slice/crio-4d9f37b3cd00243e04829fcc12374461776205ce1f22400e44be2672edcbe9a5 WatchSource:0}: Error finding container 4d9f37b3cd00243e04829fcc12374461776205ce1f22400e44be2672edcbe9a5: Status 404 returned error can't find the container with id 4d9f37b3cd00243e04829fcc12374461776205ce1f22400e44be2672edcbe9a5
Jul 28 20:30:00.394124 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.394101    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8"]
Jul 28 20:30:00.396542 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:30:00.396495    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod296f127e_f334_4a65_8b60_bb34285e57e7.slice/crio-1ec3bdcf932b37fc4c843f4b0c8e345dbf38b2efd7b4a30b51cece3c83bc10ec WatchSource:0}: Error finding container 1ec3bdcf932b37fc4c843f4b0c8e345dbf38b2efd7b4a30b51cece3c83bc10ec: Status 404 returned error can't find the container with id 1ec3bdcf932b37fc4c843f4b0c8e345dbf38b2efd7b4a30b51cece3c83bc10ec
Jul 28 20:30:00.578529 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.578506    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc"]
Jul 28 20:30:00.580683 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:30:00.580650    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b823a9a_7579_4d2a_af69_f47c15805cb6.slice/crio-5fecc6dde702de811e7a30a2434359a2b2deb9453748b02e837ccf78e1419cd5 WatchSource:0}: Error finding container 5fecc6dde702de811e7a30a2434359a2b2deb9453748b02e837ccf78e1419cd5: Status 404 returned error can't find the container with id 5fecc6dde702de811e7a30a2434359a2b2deb9453748b02e837ccf78e1419cd5
Jul 28 20:30:00.864243 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.864211    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8" event={"ID":"296f127e-f334-4a65-8b60-bb34285e57e7","Type":"ContainerStarted","Data":"79e42157b9ab8bdfac7a859fc25b5eacb2cfeaf09ad12df7634939ccb555f800"}
Jul 28 20:30:00.864410 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.864263    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8" event={"ID":"296f127e-f334-4a65-8b60-bb34285e57e7","Type":"ContainerStarted","Data":"1ec3bdcf932b37fc4c843f4b0c8e345dbf38b2efd7b4a30b51cece3c83bc10ec"}
Jul 28 20:30:00.865484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.865453    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc" event={"ID":"4b823a9a-7579-4d2a-af69-f47c15805cb6","Type":"ContainerStarted","Data":"5fecc6dde702de811e7a30a2434359a2b2deb9453748b02e837ccf78e1419cd5"}
Jul 28 20:30:00.866828 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.866806    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l" event={"ID":"5f451faa-6e75-48b4-be4a-cefe3327b9bb","Type":"ContainerStarted","Data":"7e2619cb507ab41ffde0793e74c81202dc2843a8c079602ed195e95dd5a739df"}
Jul 28 20:30:00.866910 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.866843    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l" event={"ID":"5f451faa-6e75-48b4-be4a-cefe3327b9bb","Type":"ContainerStarted","Data":"4d9f37b3cd00243e04829fcc12374461776205ce1f22400e44be2672edcbe9a5"}
Jul 28 20:30:00.878542 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.878503    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8" podStartSLOduration=0.878489561 podStartE2EDuration="878.489561ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:00.877566392 +0000 UTC m=+2582.058083449" watchObservedRunningTime="2026-07-28 20:30:00.878489561 +0000 UTC m=+2582.059006616"
Jul 28 20:30:00.892293 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:00.892251    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l" podStartSLOduration=0.892238331 podStartE2EDuration="892.238331ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:00.890594838 +0000 UTC m=+2582.071111894" watchObservedRunningTime="2026-07-28 20:30:00.892238331 +0000 UTC m=+2582.072755429"
Jul 28 20:30:01.873614 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:01.873515    2569 generic.go:356] "Generic (PLEG): container finished" podID="296f127e-f334-4a65-8b60-bb34285e57e7" containerID="79e42157b9ab8bdfac7a859fc25b5eacb2cfeaf09ad12df7634939ccb555f800" exitCode=0
Jul 28 20:30:01.873614 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:01.873603    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8" event={"ID":"296f127e-f334-4a65-8b60-bb34285e57e7","Type":"ContainerDied","Data":"79e42157b9ab8bdfac7a859fc25b5eacb2cfeaf09ad12df7634939ccb555f800"}
Jul 28 20:30:01.874988 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:01.874965    2569 generic.go:356] "Generic (PLEG): container finished" podID="4b823a9a-7579-4d2a-af69-f47c15805cb6" containerID="de83bad86cb4619beab68085e66bf1aac904e3398f16bd94828a3376e1e8d990" exitCode=0
Jul 28 20:30:01.875091 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:01.875048    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc" event={"ID":"4b823a9a-7579-4d2a-af69-f47c15805cb6","Type":"ContainerDied","Data":"de83bad86cb4619beab68085e66bf1aac904e3398f16bd94828a3376e1e8d990"}
Jul 28 20:30:01.876576 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:01.876554    2569 generic.go:356] "Generic (PLEG): container finished" podID="5f451faa-6e75-48b4-be4a-cefe3327b9bb" containerID="7e2619cb507ab41ffde0793e74c81202dc2843a8c079602ed195e95dd5a739df" exitCode=0
Jul 28 20:30:01.876718 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:01.876607    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l" event={"ID":"5f451faa-6e75-48b4-be4a-cefe3327b9bb","Type":"ContainerDied","Data":"7e2619cb507ab41ffde0793e74c81202dc2843a8c079602ed195e95dd5a739df"}
Jul 28 20:30:03.026782 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.026749    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc"
Jul 28 20:30:03.073247 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.073221    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l"
Jul 28 20:30:03.076503 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.076475    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8"
Jul 28 20:30:03.888893 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.888859    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8"
Jul 28 20:30:03.889070 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.888861    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754510-mnbx8" event={"ID":"296f127e-f334-4a65-8b60-bb34285e57e7","Type":"ContainerDied","Data":"1ec3bdcf932b37fc4c843f4b0c8e345dbf38b2efd7b4a30b51cece3c83bc10ec"}
Jul 28 20:30:03.889070 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.888989    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ec3bdcf932b37fc4c843f4b0c8e345dbf38b2efd7b4a30b51cece3c83bc10ec"
Jul 28 20:30:03.890247 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.890225    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc"
Jul 28 20:30:03.890410 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.890256    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc" event={"ID":"4b823a9a-7579-4d2a-af69-f47c15805cb6","Type":"ContainerDied","Data":"5fecc6dde702de811e7a30a2434359a2b2deb9453748b02e837ccf78e1419cd5"}
Jul 28 20:30:03.890410 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.890318    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fecc6dde702de811e7a30a2434359a2b2deb9453748b02e837ccf78e1419cd5"
Jul 28 20:30:03.891898 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.891879    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l"
Jul 28 20:30:03.891984 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.891912    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l" event={"ID":"5f451faa-6e75-48b4-be4a-cefe3327b9bb","Type":"ContainerDied","Data":"4d9f37b3cd00243e04829fcc12374461776205ce1f22400e44be2672edcbe9a5"}
Jul 28 20:30:03.891984 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:03.891944    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d9f37b3cd00243e04829fcc12374461776205ce1f22400e44be2672edcbe9a5"
Jul 28 20:30:46.974850 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:46.974815    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:47.110513 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:30:47.110483    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:59.451978 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:31:59.451911    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:31:59.461802 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:31:59.461785    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:31:59.554524 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:31:59.554496    2569 scope.go:117] "RemoveContainer" containerID="b1af6c83337c576f613c26e64ff1b7fc0bb3d8bcd4959415a612d9489b697de1"
Jul 28 20:31:59.563665 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:31:59.563640    2569 scope.go:117] "RemoveContainer" containerID="ffbccd50ad4695751db124737fa25d4db32cc637d2263d3ea3670ea152f65a2d"
Jul 28 20:32:42.108054 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:32:42.108016    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:48.047142 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:32:48.047110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:59.587834 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:32:59.587801    2569 scope.go:117] "RemoveContainer" containerID="96f9ead6925fc8f632465728a555e0c78e032a0f1171f2d38bb45f951ae0256c"
Jul 28 20:33:39.780421 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:39.780337    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"]
Jul 28 20:33:39.854901 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:39.854869    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:40.596291 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:40.596251    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"]
Jul 28 20:33:40.606424 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:40.606391    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:54.799068 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:54.799031    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm" podUID="7e800be2-bfcc-454c-9f36-e4dabfbf54c3" containerName="main" containerID="cri-o://20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212" gracePeriod=45
Jul 28 20:33:55.052802 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.052749    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:33:55.079099 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.079066    2569 generic.go:356] "Generic (PLEG): container finished" podID="7e800be2-bfcc-454c-9f36-e4dabfbf54c3" containerID="20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212" exitCode=0
Jul 28 20:33:55.079232 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.079100    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"
Jul 28 20:33:55.079232 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.079147    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm" event={"ID":"7e800be2-bfcc-454c-9f36-e4dabfbf54c3","Type":"ContainerDied","Data":"20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212"}
Jul 28 20:33:55.079232 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.079185    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm" event={"ID":"7e800be2-bfcc-454c-9f36-e4dabfbf54c3","Type":"ContainerDied","Data":"b81a8475db9f7d1db038fb1e7ced90e718525afe2b08fd272015addcf77a5f3c"}
Jul 28 20:33:55.079232 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.079202    2569 scope.go:117] "RemoveContainer" containerID="20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212"
Jul 28 20:33:55.087511 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.087485    2569 scope.go:117] "RemoveContainer" containerID="20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212"
Jul 28 20:33:55.087778 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:33:55.087756    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212\": container with ID starting with 20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212 not found: ID does not exist" containerID="20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212"
Jul 28 20:33:55.087827 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.087790    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212"} err="failed to get container status \"20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212\": rpc error: code = NotFound desc = could not find container \"20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212\": container with ID starting with 20f80a993800ad462e3f56fe1b40d5afed3fb47ee9aaf3fd56dd0b0121b1a212 not found: ID does not exist"
Jul 28 20:33:55.128193 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128171    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-home\") pod \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") "
Jul 28 20:33:55.128298 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128220    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q49sw\" (UniqueName: \"kubernetes.io/projected/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-kube-api-access-q49sw\") pod \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") "
Jul 28 20:33:55.128298 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128251    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tmp-dir\") pod \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") "
Jul 28 20:33:55.128391 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128342    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tls-certs\") pod \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") "
Jul 28 20:33:55.128391 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128363    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-model-cache\") pod \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") "
Jul 28 20:33:55.128468 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128392    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-dshm\") pod \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\" (UID: \"7e800be2-bfcc-454c-9f36-e4dabfbf54c3\") "
Jul 28 20:33:55.128468 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128437    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-home" (OuterVolumeSpecName: "home") pod "7e800be2-bfcc-454c-9f36-e4dabfbf54c3" (UID: "7e800be2-bfcc-454c-9f36-e4dabfbf54c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:33:55.128613 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128574    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e800be2-bfcc-454c-9f36-e4dabfbf54c3" (UID: "7e800be2-bfcc-454c-9f36-e4dabfbf54c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:33:55.128683 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128652    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e800be2-bfcc-454c-9f36-e4dabfbf54c3" (UID: "7e800be2-bfcc-454c-9f36-e4dabfbf54c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:33:55.128760 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128746    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:55.128818 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128766    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:55.128818 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.128779    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:55.130452 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.130432    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-kube-api-access-q49sw" (OuterVolumeSpecName: "kube-api-access-q49sw") pod "7e800be2-bfcc-454c-9f36-e4dabfbf54c3" (UID: "7e800be2-bfcc-454c-9f36-e4dabfbf54c3"). InnerVolumeSpecName "kube-api-access-q49sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:33:55.130543 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.130465    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e800be2-bfcc-454c-9f36-e4dabfbf54c3" (UID: "7e800be2-bfcc-454c-9f36-e4dabfbf54c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:33:55.130543 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.130465    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-dshm" (OuterVolumeSpecName: "dshm") pod "7e800be2-bfcc-454c-9f36-e4dabfbf54c3" (UID: "7e800be2-bfcc-454c-9f36-e4dabfbf54c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:33:55.229375 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.229350    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q49sw\" (UniqueName: \"kubernetes.io/projected/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-kube-api-access-q49sw\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:55.229375 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.229374    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:55.229521 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.229383    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e800be2-bfcc-454c-9f36-e4dabfbf54c3-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:55.378362 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:33:55.378335    2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e800be2_bfcc_454c_9f36_e4dabfbf54c3.slice/crio-b81a8475db9f7d1db038fb1e7ced90e718525afe2b08fd272015addcf77a5f3c\": RecentStats: unable to find data in memory cache]"
Jul 28 20:33:55.393637 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.393603    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"]
Jul 28 20:33:55.396964 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.396943    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/auth-test-model-ed54db-kserve-5b4d4d8859-lpvzm"]
Jul 28 20:33:55.622680 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.622643    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj" podUID="cd5e2766-2447-488c-9441-7b5647be4527" containerName="main" containerID="cri-o://2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490" gracePeriod=45
Jul 28 20:33:55.877373 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.877320    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:33:55.937227 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.937193    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-dshm\") pod \"cd5e2766-2447-488c-9441-7b5647be4527\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") "
Jul 28 20:33:55.937382 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.937261    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-tmp-dir\") pod \"cd5e2766-2447-488c-9441-7b5647be4527\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") "
Jul 28 20:33:55.937382 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.937333    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmnz8\" (UniqueName: \"kubernetes.io/projected/cd5e2766-2447-488c-9441-7b5647be4527-kube-api-access-zmnz8\") pod \"cd5e2766-2447-488c-9441-7b5647be4527\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") "
Jul 28 20:33:55.937382 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.937357    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2766-2447-488c-9441-7b5647be4527-tls-certs\") pod \"cd5e2766-2447-488c-9441-7b5647be4527\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") "
Jul 28 20:33:55.937553 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.937428    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-home\") pod \"cd5e2766-2447-488c-9441-7b5647be4527\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") "
Jul 28 20:33:55.937553 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.937475    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-model-cache\") pod \"cd5e2766-2447-488c-9441-7b5647be4527\" (UID: \"cd5e2766-2447-488c-9441-7b5647be4527\") "
Jul 28 20:33:55.937553 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.937516    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd5e2766-2447-488c-9441-7b5647be4527" (UID: "cd5e2766-2447-488c-9441-7b5647be4527"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:33:55.937720 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.937696    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-home" (OuterVolumeSpecName: "home") pod "cd5e2766-2447-488c-9441-7b5647be4527" (UID: "cd5e2766-2447-488c-9441-7b5647be4527"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:33:55.937832 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.937813    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd5e2766-2447-488c-9441-7b5647be4527" (UID: "cd5e2766-2447-488c-9441-7b5647be4527"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:33:55.937887 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.937853    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:55.937887 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.937872    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:55.939471 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.939451    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd5e2766-2447-488c-9441-7b5647be4527-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd5e2766-2447-488c-9441-7b5647be4527" (UID: "cd5e2766-2447-488c-9441-7b5647be4527"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:33:55.939529 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.939470    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-dshm" (OuterVolumeSpecName: "dshm") pod "cd5e2766-2447-488c-9441-7b5647be4527" (UID: "cd5e2766-2447-488c-9441-7b5647be4527"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:33:55.939671 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:55.939655    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd5e2766-2447-488c-9441-7b5647be4527-kube-api-access-zmnz8" (OuterVolumeSpecName: "kube-api-access-zmnz8") pod "cd5e2766-2447-488c-9441-7b5647be4527" (UID: "cd5e2766-2447-488c-9441-7b5647be4527"). InnerVolumeSpecName "kube-api-access-zmnz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:33:56.038895 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.038869    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:56.038895 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.038892    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd5e2766-2447-488c-9441-7b5647be4527-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:56.039027 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.038905    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmnz8\" (UniqueName: \"kubernetes.io/projected/cd5e2766-2447-488c-9441-7b5647be4527-kube-api-access-zmnz8\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:56.039027 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.038919    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd5e2766-2447-488c-9441-7b5647be4527-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:33:56.086705 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.086683    2569 generic.go:356] "Generic (PLEG): container finished" podID="cd5e2766-2447-488c-9441-7b5647be4527" containerID="2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490" exitCode=0
Jul 28 20:33:56.086790 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.086726    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"
Jul 28 20:33:56.086790 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.086730    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj" event={"ID":"cd5e2766-2447-488c-9441-7b5647be4527","Type":"ContainerDied","Data":"2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490"}
Jul 28 20:33:56.086790 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.086764    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj" event={"ID":"cd5e2766-2447-488c-9441-7b5647be4527","Type":"ContainerDied","Data":"803c4e6e5a36eb24fdb7f05dda83ad0131034845a05ec6abbde0ff49a344f7e8"}
Jul 28 20:33:56.086790 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.086781    2569 scope.go:117] "RemoveContainer" containerID="2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490"
Jul 28 20:33:56.095243 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.095228    2569 scope.go:117] "RemoveContainer" containerID="2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490"
Jul 28 20:33:56.095487 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:33:56.095465    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490\": container with ID starting with 2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490 not found: ID does not exist" containerID="2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490"
Jul 28 20:33:56.095574 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.095497    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490"} err="failed to get container status \"2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490\": rpc error: code = NotFound desc = could not find container \"2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490\": container with ID starting with 2d61fbbf01766c0bf25f678ed0a27fd08515bb09e7db10c890cf0e2c2f727490 not found: ID does not exist"
Jul 28 20:33:56.111917 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.111893    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"]
Jul 28 20:33:56.115671 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:56.115642    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/auth-test-model-1231ba-kserve-59bf4696c5-rnczj"]
Jul 28 20:33:57.369302 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:57.369221    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e800be2-bfcc-454c-9f36-e4dabfbf54c3" path="/var/lib/kubelet/pods/7e800be2-bfcc-454c-9f36-e4dabfbf54c3/volumes"
Jul 28 20:33:57.372296 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:57.370399    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd5e2766-2447-488c-9441-7b5647be4527" path="/var/lib/kubelet/pods/cd5e2766-2447-488c-9441-7b5647be4527/volumes"
Jul 28 20:33:59.607811 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:33:59.607779    2569 scope.go:117] "RemoveContainer" containerID="144a31862283e1a3db1f3e59463c2060576ecdfc22b091d823081e7572aed78c"
Jul 28 20:34:06.791932 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.791900    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"]
Jul 28 20:34:06.792325 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792270    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b823a9a-7579-4d2a-af69-f47c15805cb6" containerName="cleanup"
Jul 28 20:34:06.792325 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792295    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b823a9a-7579-4d2a-af69-f47c15805cb6" containerName="cleanup"
Jul 28 20:34:06.792325 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792316    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd5e2766-2447-488c-9441-7b5647be4527" containerName="main"
Jul 28 20:34:06.792325 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792322    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd5e2766-2447-488c-9441-7b5647be4527" containerName="main"
Jul 28 20:34:06.792484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792333    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f451faa-6e75-48b4-be4a-cefe3327b9bb" containerName="cleanup"
Jul 28 20:34:06.792484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792339    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f451faa-6e75-48b4-be4a-cefe3327b9bb" containerName="cleanup"
Jul 28 20:34:06.792484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792344    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="296f127e-f334-4a65-8b60-bb34285e57e7" containerName="cleanup"
Jul 28 20:34:06.792484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792349    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="296f127e-f334-4a65-8b60-bb34285e57e7" containerName="cleanup"
Jul 28 20:34:06.792484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792360    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e800be2-bfcc-454c-9f36-e4dabfbf54c3" containerName="main"
Jul 28 20:34:06.792484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792365    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e800be2-bfcc-454c-9f36-e4dabfbf54c3" containerName="main"
Jul 28 20:34:06.792484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792431    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f451faa-6e75-48b4-be4a-cefe3327b9bb" containerName="cleanup"
Jul 28 20:34:06.792484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792440    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b823a9a-7579-4d2a-af69-f47c15805cb6" containerName="cleanup"
Jul 28 20:34:06.792484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792448    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e800be2-bfcc-454c-9f36-e4dabfbf54c3" containerName="main"
Jul 28 20:34:06.792484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792454    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd5e2766-2447-488c-9441-7b5647be4527" containerName="main"
Jul 28 20:34:06.792484 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.792460    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="296f127e-f334-4a65-8b60-bb34285e57e7" containerName="cleanup"
Jul 28 20:34:06.796919 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.796898    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:06.799552 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.799512    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ed54db\"/\"default-dockercfg-l7hfk\""
Jul 28 20:34:06.799673 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.799587    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ed54db\"/\"kube-root-ca.crt\""
Jul 28 20:34:06.799739 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.799682    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ed54db\"/\"sub-test-model-ed54db-kserve-self-signed-certs\""
Jul 28 20:34:06.799797 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.799752    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ed54db\"/\"openshift-service-ca.crt\""
Jul 28 20:34:06.808179 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.808154    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"]
Jul 28 20:34:06.828517 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.828484    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:06.937325 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.937273    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z4nv\" (UniqueName: \"kubernetes.io/projected/98206f73-c32f-41f9-aac8-79057fb5944d-kube-api-access-6z4nv\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:06.937482 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.937345    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98206f73-c32f-41f9-aac8-79057fb5944d-tls-certs\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:06.937482 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.937393    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-tmp-dir\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:06.937482 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.937459    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-dshm\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:06.937594 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.937481    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-home\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:06.937594 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:06.937520    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-model-cache\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.038542 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.038497    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-model-cache\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.038696 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.038575    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z4nv\" (UniqueName: \"kubernetes.io/projected/98206f73-c32f-41f9-aac8-79057fb5944d-kube-api-access-6z4nv\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.038696 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.038621    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98206f73-c32f-41f9-aac8-79057fb5944d-tls-certs\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.038696 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.038681    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-tmp-dir\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.038833 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.038744    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-dshm\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.038833 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.038774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-home\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.038980 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.038958    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-model-cache\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.039131 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.039111    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-tmp-dir\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.039195 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.039118    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-home\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.040904 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.040880    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-dshm\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.041089 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.041071    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98206f73-c32f-41f9-aac8-79057fb5944d-tls-certs\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.047489 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.047428    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z4nv\" (UniqueName: \"kubernetes.io/projected/98206f73-c32f-41f9-aac8-79057fb5944d-kube-api-access-6z4nv\") pod \"sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") " pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.108449 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.108423    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:07.245938 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:07.245912    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"]
Jul 28 20:34:07.248301 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:34:07.248256    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98206f73_c32f_41f9_aac8_79057fb5944d.slice/crio-4def226f01035ecd5680d34026b1c6c58c6a63158e6612dd4a3c69bec0cf12c5 WatchSource:0}: Error finding container 4def226f01035ecd5680d34026b1c6c58c6a63158e6612dd4a3c69bec0cf12c5: Status 404 returned error can't find the container with id 4def226f01035ecd5680d34026b1c6c58c6a63158e6612dd4a3c69bec0cf12c5
Jul 28 20:34:08.145714 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:08.145679    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp" event={"ID":"98206f73-c32f-41f9-aac8-79057fb5944d","Type":"ContainerStarted","Data":"8343bc0a94604b7c5ae9f2da825ee5fd5a584b36a513f53ad938e9202160e3ba"}
Jul 28 20:34:08.146117 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:08.145722    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp" event={"ID":"98206f73-c32f-41f9-aac8-79057fb5944d","Type":"ContainerStarted","Data":"4def226f01035ecd5680d34026b1c6c58c6a63158e6612dd4a3c69bec0cf12c5"}
Jul 28 20:34:08.165510 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:08.165460    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp" podStartSLOduration=2.165441875 podStartE2EDuration="2.165441875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:34:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:34:08.165434422 +0000 UTC m=+2829.345951478" watchObservedRunningTime="2026-07-28 20:34:08.165441875 +0000 UTC m=+2829.345958933"
Jul 28 20:34:17.109369 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:17.109333    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:17.109901 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:17.109409    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:17.122268 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:17.122247    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:17.207171 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:17.207147    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:34:29.385065 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.385029    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"]
Jul 28 20:34:29.388842 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.388818    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.391097 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.391071    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"openshift-service-ca.crt\""
Jul 28 20:34:29.391755 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.391720    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"sub-test-model-1231ba-kserve-self-signed-certs\""
Jul 28 20:34:29.391755 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.391729    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"default-dockercfg-l6d6k\""
Jul 28 20:34:29.391926 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.391790    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"kube-root-ca.crt\""
Jul 28 20:34:29.399619 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.399595    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"]
Jul 28 20:34:29.422733 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.422710    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:29.450211 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.450189    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49dfeb75-a421-4bf7-84c9-a7e750405932-tls-certs\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.450359 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.450226    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-tmp-dir\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.450359 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.450260    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-dshm\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.450482 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.450353    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rsjw\" (UniqueName: \"kubernetes.io/projected/49dfeb75-a421-4bf7-84c9-a7e750405932-kube-api-access-8rsjw\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.450482 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.450434    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-home\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.450578 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.450488    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-model-cache\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.551781 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.551759    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-home\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.551913 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.551797    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-model-cache\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.551913 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.551828    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49dfeb75-a421-4bf7-84c9-a7e750405932-tls-certs\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.551913 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.551856    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-tmp-dir\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.551913 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.551891    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-dshm\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.552136 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.551938    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rsjw\" (UniqueName: \"kubernetes.io/projected/49dfeb75-a421-4bf7-84c9-a7e750405932-kube-api-access-8rsjw\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.552198 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.552142    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-model-cache\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.552253 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.552206    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-home\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.552340 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.552322    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-tmp-dir\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.554165 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.554142    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-dshm\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.554384 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.554370    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49dfeb75-a421-4bf7-84c9-a7e750405932-tls-certs\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.559874 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.559855    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rsjw\" (UniqueName: \"kubernetes.io/projected/49dfeb75-a421-4bf7-84c9-a7e750405932-kube-api-access-8rsjw\") pod \"sub-test-model-1231ba-kserve-5594555d67-xwgxx\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") " pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.699786 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.699762    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:29.836904 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:29.836876    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"]
Jul 28 20:34:29.838636 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:34:29.838605    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49dfeb75_a421_4bf7_84c9_a7e750405932.slice/crio-214056cf413a883e87be6999284260c666c118de755ef8dbd7b6a0a2feae9d7c WatchSource:0}: Error finding container 214056cf413a883e87be6999284260c666c118de755ef8dbd7b6a0a2feae9d7c: Status 404 returned error can't find the container with id 214056cf413a883e87be6999284260c666c118de755ef8dbd7b6a0a2feae9d7c
Jul 28 20:34:30.261163 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:30.261122    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx" event={"ID":"49dfeb75-a421-4bf7-84c9-a7e750405932","Type":"ContainerStarted","Data":"38ed7cd4b06406075bdfa6e20f381ac18f79c9076c628cfe0493447ead09ea3c"}
Jul 28 20:34:30.261378 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:30.261172    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx" event={"ID":"49dfeb75-a421-4bf7-84c9-a7e750405932","Type":"ContainerStarted","Data":"214056cf413a883e87be6999284260c666c118de755ef8dbd7b6a0a2feae9d7c"}
Jul 28 20:34:30.282437 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:30.282390    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx" podStartSLOduration=1.282376266 podStartE2EDuration="1.282376266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:34:30.28009204 +0000 UTC m=+2851.460609096" watchObservedRunningTime="2026-07-28 20:34:30.282376266 +0000 UTC m=+2851.462893369"
Jul 28 20:34:39.700821 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:39.700767    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:39.700821 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:39.700836    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:39.713720 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:39.713692    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:34:40.326867 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:34:40.326831    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:35:35.487897 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:35.487817    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"]
Jul 28 20:35:35.508144 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:35.508116    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:35:36.375195 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:36.375161    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"]
Jul 28 20:35:36.398443 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:36.398417    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:35:50.506571 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.506524    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp" podUID="98206f73-c32f-41f9-aac8-79057fb5944d" containerName="main" containerID="cri-o://8343bc0a94604b7c5ae9f2da825ee5fd5a584b36a513f53ad938e9202160e3ba" gracePeriod=45
Jul 28 20:35:50.658145 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.658110    2569 generic.go:356] "Generic (PLEG): container finished" podID="98206f73-c32f-41f9-aac8-79057fb5944d" containerID="8343bc0a94604b7c5ae9f2da825ee5fd5a584b36a513f53ad938e9202160e3ba" exitCode=0
Jul 28 20:35:50.658313 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.658185    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp" event={"ID":"98206f73-c32f-41f9-aac8-79057fb5944d","Type":"ContainerDied","Data":"8343bc0a94604b7c5ae9f2da825ee5fd5a584b36a513f53ad938e9202160e3ba"}
Jul 28 20:35:50.761568 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.761517    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:35:50.859446 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.859417    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-home\") pod \"98206f73-c32f-41f9-aac8-79057fb5944d\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") "
Jul 28 20:35:50.859582 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.859482    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z4nv\" (UniqueName: \"kubernetes.io/projected/98206f73-c32f-41f9-aac8-79057fb5944d-kube-api-access-6z4nv\") pod \"98206f73-c32f-41f9-aac8-79057fb5944d\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") "
Jul 28 20:35:50.859582 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.859503    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-dshm\") pod \"98206f73-c32f-41f9-aac8-79057fb5944d\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") "
Jul 28 20:35:50.859582 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.859522    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-model-cache\") pod \"98206f73-c32f-41f9-aac8-79057fb5944d\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") "
Jul 28 20:35:50.859582 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.859570    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-tmp-dir\") pod \"98206f73-c32f-41f9-aac8-79057fb5944d\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") "
Jul 28 20:35:50.859772 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.859603    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98206f73-c32f-41f9-aac8-79057fb5944d-tls-certs\") pod \"98206f73-c32f-41f9-aac8-79057fb5944d\" (UID: \"98206f73-c32f-41f9-aac8-79057fb5944d\") "
Jul 28 20:35:50.859772 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.859712    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-home" (OuterVolumeSpecName: "home") pod "98206f73-c32f-41f9-aac8-79057fb5944d" (UID: "98206f73-c32f-41f9-aac8-79057fb5944d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:35:50.859890 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.859820    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-model-cache" (OuterVolumeSpecName: "model-cache") pod "98206f73-c32f-41f9-aac8-79057fb5944d" (UID: "98206f73-c32f-41f9-aac8-79057fb5944d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:35:50.859890 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.859840    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98206f73-c32f-41f9-aac8-79057fb5944d" (UID: "98206f73-c32f-41f9-aac8-79057fb5944d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:35:50.860022 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.860005    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:50.860062 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.860028    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:50.860062 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.860041    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:50.861606 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.861580    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-dshm" (OuterVolumeSpecName: "dshm") pod "98206f73-c32f-41f9-aac8-79057fb5944d" (UID: "98206f73-c32f-41f9-aac8-79057fb5944d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:35:50.861606 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.861593    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98206f73-c32f-41f9-aac8-79057fb5944d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98206f73-c32f-41f9-aac8-79057fb5944d" (UID: "98206f73-c32f-41f9-aac8-79057fb5944d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:35:50.861606 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.861596    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98206f73-c32f-41f9-aac8-79057fb5944d-kube-api-access-6z4nv" (OuterVolumeSpecName: "kube-api-access-6z4nv") pod "98206f73-c32f-41f9-aac8-79057fb5944d" (UID: "98206f73-c32f-41f9-aac8-79057fb5944d"). InnerVolumeSpecName "kube-api-access-6z4nv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:35:50.961021 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.960998    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z4nv\" (UniqueName: \"kubernetes.io/projected/98206f73-c32f-41f9-aac8-79057fb5944d-kube-api-access-6z4nv\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:50.961021 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.961018    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98206f73-c32f-41f9-aac8-79057fb5944d-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:50.961161 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:50.961028    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98206f73-c32f-41f9-aac8-79057fb5944d-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:51.166321 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.166247    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"]
Jul 28 20:35:51.166678 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.166665    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98206f73-c32f-41f9-aac8-79057fb5944d" containerName="main"
Jul 28 20:35:51.166721 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.166680    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="98206f73-c32f-41f9-aac8-79057fb5944d" containerName="main"
Jul 28 20:35:51.166756 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.166745    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="98206f73-c32f-41f9-aac8-79057fb5944d" containerName="main"
Jul 28 20:35:51.170565 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.170546    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.172454 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.172437    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ed54db\"/\"rate-test-model-ed54db-kserve-self-signed-certs\""
Jul 28 20:35:51.179305 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.179259    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"]
Jul 28 20:35:51.191122 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.191098    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:35:51.263875 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.263848    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-home\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.263996 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.263894    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmzpc\" (UniqueName: \"kubernetes.io/projected/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-kube-api-access-kmzpc\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.263996 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.263916    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-dshm\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.264113 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.264018    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-model-cache\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.264170 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.264114    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tmp-dir\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.264170 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.264144    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tls-certs\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.364922 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.364898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-home\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.365031 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.364945    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmzpc\" (UniqueName: \"kubernetes.io/projected/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-kube-api-access-kmzpc\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.365031 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.364973    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-dshm\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.365031 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.365005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-model-cache\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.365210 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.365167    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tmp-dir\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.365210 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.365198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tls-certs\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.365347 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.365328    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-home\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.365405 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.365351    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-model-cache\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.365518 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.365499    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tmp-dir\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.367225 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.367201    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-dshm\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.367524 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.367488    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tls-certs\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.372894 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.372875    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmzpc\" (UniqueName: \"kubernetes.io/projected/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-kube-api-access-kmzpc\") pod \"rate-test-model-ed54db-kserve-86488b888c-zhhf4\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") " pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.393542 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.393519    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx" podUID="49dfeb75-a421-4bf7-84c9-a7e750405932" containerName="main" containerID="cri-o://38ed7cd4b06406075bdfa6e20f381ac18f79c9076c628cfe0493447ead09ea3c" gracePeriod=45
Jul 28 20:35:51.481350 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.481311    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:35:51.665077 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.665052    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"
Jul 28 20:35:51.665486 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.665048    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp" event={"ID":"98206f73-c32f-41f9-aac8-79057fb5944d","Type":"ContainerDied","Data":"4def226f01035ecd5680d34026b1c6c58c6a63158e6612dd4a3c69bec0cf12c5"}
Jul 28 20:35:51.665486 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.665170    2569 scope.go:117] "RemoveContainer" containerID="8343bc0a94604b7c5ae9f2da825ee5fd5a584b36a513f53ad938e9202160e3ba"
Jul 28 20:35:51.666486 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.666462    2569 generic.go:356] "Generic (PLEG): container finished" podID="49dfeb75-a421-4bf7-84c9-a7e750405932" containerID="38ed7cd4b06406075bdfa6e20f381ac18f79c9076c628cfe0493447ead09ea3c" exitCode=0
Jul 28 20:35:51.666590 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.666542    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx" event={"ID":"49dfeb75-a421-4bf7-84c9-a7e750405932","Type":"ContainerDied","Data":"38ed7cd4b06406075bdfa6e20f381ac18f79c9076c628cfe0493447ead09ea3c"}
Jul 28 20:35:51.666590 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.666575    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx" event={"ID":"49dfeb75-a421-4bf7-84c9-a7e750405932","Type":"ContainerDied","Data":"214056cf413a883e87be6999284260c666c118de755ef8dbd7b6a0a2feae9d7c"}
Jul 28 20:35:51.666590 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.666586    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="214056cf413a883e87be6999284260c666c118de755ef8dbd7b6a0a2feae9d7c"
Jul 28 20:35:51.676689 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.676673    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:35:51.686297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.686263    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"]
Jul 28 20:35:51.690337 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.690317    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/sub-test-model-ed54db-kserve-6f78fdbcff-rbqmp"]
Jul 28 20:35:51.769143 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769077    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-dshm\") pod \"49dfeb75-a421-4bf7-84c9-a7e750405932\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") "
Jul 28 20:35:51.769258 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769168    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rsjw\" (UniqueName: \"kubernetes.io/projected/49dfeb75-a421-4bf7-84c9-a7e750405932-kube-api-access-8rsjw\") pod \"49dfeb75-a421-4bf7-84c9-a7e750405932\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") "
Jul 28 20:35:51.769258 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769219    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49dfeb75-a421-4bf7-84c9-a7e750405932-tls-certs\") pod \"49dfeb75-a421-4bf7-84c9-a7e750405932\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") "
Jul 28 20:35:51.769408 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769261    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-home\") pod \"49dfeb75-a421-4bf7-84c9-a7e750405932\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") "
Jul 28 20:35:51.769408 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769332    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-tmp-dir\") pod \"49dfeb75-a421-4bf7-84c9-a7e750405932\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") "
Jul 28 20:35:51.769408 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769392    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-model-cache\") pod \"49dfeb75-a421-4bf7-84c9-a7e750405932\" (UID: \"49dfeb75-a421-4bf7-84c9-a7e750405932\") "
Jul 28 20:35:51.769574 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769517    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-home" (OuterVolumeSpecName: "home") pod "49dfeb75-a421-4bf7-84c9-a7e750405932" (UID: "49dfeb75-a421-4bf7-84c9-a7e750405932"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:35:51.769638 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769612    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "49dfeb75-a421-4bf7-84c9-a7e750405932" (UID: "49dfeb75-a421-4bf7-84c9-a7e750405932"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:35:51.769702 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769689    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-model-cache" (OuterVolumeSpecName: "model-cache") pod "49dfeb75-a421-4bf7-84c9-a7e750405932" (UID: "49dfeb75-a421-4bf7-84c9-a7e750405932"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:35:51.769829 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769779    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:51.769829 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769803    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:51.769829 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.769818    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:51.771246 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.771224    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49dfeb75-a421-4bf7-84c9-a7e750405932-kube-api-access-8rsjw" (OuterVolumeSpecName: "kube-api-access-8rsjw") pod "49dfeb75-a421-4bf7-84c9-a7e750405932" (UID: "49dfeb75-a421-4bf7-84c9-a7e750405932"). InnerVolumeSpecName "kube-api-access-8rsjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:35:51.771365 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.771350    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-dshm" (OuterVolumeSpecName: "dshm") pod "49dfeb75-a421-4bf7-84c9-a7e750405932" (UID: "49dfeb75-a421-4bf7-84c9-a7e750405932"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:35:51.771417 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.771382    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49dfeb75-a421-4bf7-84c9-a7e750405932-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "49dfeb75-a421-4bf7-84c9-a7e750405932" (UID: "49dfeb75-a421-4bf7-84c9-a7e750405932"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:35:51.826581 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.826558    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"]
Jul 28 20:35:51.828744 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:35:51.828722    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf83c241_c8a4_4d5b_8ba0_e191a4e98fb9.slice/crio-9a9bede2017dbcdb43ba730c12e5134113675173dd74cc0b5fc21cdd1cf46528 WatchSource:0}: Error finding container 9a9bede2017dbcdb43ba730c12e5134113675173dd74cc0b5fc21cdd1cf46528: Status 404 returned error can't find the container with id 9a9bede2017dbcdb43ba730c12e5134113675173dd74cc0b5fc21cdd1cf46528
Jul 28 20:35:51.870379 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.870353    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rsjw\" (UniqueName: \"kubernetes.io/projected/49dfeb75-a421-4bf7-84c9-a7e750405932-kube-api-access-8rsjw\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:51.870379 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.870377    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49dfeb75-a421-4bf7-84c9-a7e750405932-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:51.870557 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:51.870385    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49dfeb75-a421-4bf7-84c9-a7e750405932-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:35:52.673933 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:52.673895    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4" event={"ID":"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9","Type":"ContainerStarted","Data":"82a93acd53b1be3506b0e253c71d5e511b54669416050f4b7d39aaf9d14a33cb"}
Jul 28 20:35:52.674410 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:52.673944    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4" event={"ID":"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9","Type":"ContainerStarted","Data":"9a9bede2017dbcdb43ba730c12e5134113675173dd74cc0b5fc21cdd1cf46528"}
Jul 28 20:35:52.674410 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:52.673906    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"
Jul 28 20:35:52.695430 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:52.695379    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4" podStartSLOduration=1.6953626979999998 podStartE2EDuration="1.695362698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:35:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:35:52.694610678 +0000 UTC m=+2933.875127734" watchObservedRunningTime="2026-07-28 20:35:52.695362698 +0000 UTC m=+2933.875879755"
Jul 28 20:35:52.706472 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:52.706440    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"]
Jul 28 20:35:52.712510 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:52.712484    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/sub-test-model-1231ba-kserve-5594555d67-xwgxx"]
Jul 28 20:35:53.366407 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:53.366367    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49dfeb75-a421-4bf7-84c9-a7e750405932" path="/var/lib/kubelet/pods/49dfeb75-a421-4bf7-84c9-a7e750405932/volumes"
Jul 28 20:35:53.366851 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:35:53.366830    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98206f73-c32f-41f9-aac8-79057fb5944d" path="/var/lib/kubelet/pods/98206f73-c32f-41f9-aac8-79057fb5944d/volumes"
Jul 28 20:36:01.481992 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:01.481954    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:36:01.481992 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:01.482003    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:36:01.494221 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:01.494194    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:36:01.729793 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:01.729755    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:36:34.877741 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:34.877668    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"]
Jul 28 20:36:34.878188 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:34.878046    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49dfeb75-a421-4bf7-84c9-a7e750405932" containerName="main"
Jul 28 20:36:34.878188 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:34.878057    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="49dfeb75-a421-4bf7-84c9-a7e750405932" containerName="main"
Jul 28 20:36:34.878188 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:34.878136    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="49dfeb75-a421-4bf7-84c9-a7e750405932" containerName="main"
Jul 28 20:36:34.881141 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:34.881124    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:34.883905 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:34.883861    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"openshift-service-ca.crt\""
Jul 28 20:36:34.884029 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:34.883993    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"default-dockercfg-l6d6k\""
Jul 28 20:36:34.884029 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:34.883993    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"rate-test-model-1231ba-kserve-self-signed-certs\""
Jul 28 20:36:34.884198 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:34.884095    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1231ba\"/\"kube-root-ca.crt\""
Jul 28 20:36:34.889056 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:34.889035    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"]
Jul 28 20:36:34.981985 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:34.981959    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:36:35.038085 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.038058    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-model-cache\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.038213 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.038096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-tmp-dir\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.038213 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.038138    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-home\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.038213 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.038181    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5801b420-8258-40fa-a796-60e10d46f9db-tls-certs\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.038367 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.038240    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-dshm\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.038367 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.038264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqvmr\" (UniqueName: \"kubernetes.io/projected/5801b420-8258-40fa-a796-60e10d46f9db-kube-api-access-jqvmr\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.139651 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.139586    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-dshm\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.139651 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.139616    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqvmr\" (UniqueName: \"kubernetes.io/projected/5801b420-8258-40fa-a796-60e10d46f9db-kube-api-access-jqvmr\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.139800 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.139752    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-model-cache\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.139800 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.139795    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-tmp-dir\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.139900 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.139840    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-home\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.139900 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.139893    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5801b420-8258-40fa-a796-60e10d46f9db-tls-certs\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.140166 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.140143    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-model-cache\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.140233 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.140163    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-tmp-dir\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.140319 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.140241    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-home\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.141760 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.141742    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-dshm\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.142151 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.142135    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5801b420-8258-40fa-a796-60e10d46f9db-tls-certs\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.147099 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.147076    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqvmr\" (UniqueName: \"kubernetes.io/projected/5801b420-8258-40fa-a796-60e10d46f9db-kube-api-access-jqvmr\") pod \"rate-test-model-1231ba-kserve-67cbbb88c-6rth2\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") " pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.191872 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.191849    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:35.321257 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.321232    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"]
Jul 28 20:36:35.324315 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:36:35.324271    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5801b420_8258_40fa_a796_60e10d46f9db.slice/crio-cb10387e33e68c3d58a71b46cdbc5c153d18528f8161ecb9ff17e216b4a76e6e WatchSource:0}: Error finding container cb10387e33e68c3d58a71b46cdbc5c153d18528f8161ecb9ff17e216b4a76e6e: Status 404 returned error can't find the container with id cb10387e33e68c3d58a71b46cdbc5c153d18528f8161ecb9ff17e216b4a76e6e
Jul 28 20:36:35.892610 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.892570    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2" event={"ID":"5801b420-8258-40fa-a796-60e10d46f9db","Type":"ContainerStarted","Data":"b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56"}
Jul 28 20:36:35.892610 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.892613    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2" event={"ID":"5801b420-8258-40fa-a796-60e10d46f9db","Type":"ContainerStarted","Data":"cb10387e33e68c3d58a71b46cdbc5c153d18528f8161ecb9ff17e216b4a76e6e"}
Jul 28 20:36:35.913557 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:35.913502    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2" podStartSLOduration=1.913484961 podStartE2EDuration="1.913484961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:36:35.911087607 +0000 UTC m=+2977.091604662" watchObservedRunningTime="2026-07-28 20:36:35.913484961 +0000 UTC m=+2977.094002018"
Jul 28 20:36:45.192899 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:45.192862    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:45.193427 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:45.192911    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:45.205502 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:45.205473    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:45.955629 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:45.955599    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:36:59.485264 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:59.485231    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:36:59.495115 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:36:59.495089    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:37:05.900592 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:05.900533    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:37:53.076009 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:53.075929    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kgwk"]
Jul 28 20:37:53.079754 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:53.079732    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk"
Jul 28 20:37:53.081758 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:53.081734    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-446lt\""
Jul 28 20:37:53.087271 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:53.087211    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kgwk"]
Jul 28 20:37:53.210038 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:53.210005    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbhsm\" (UniqueName: \"kubernetes.io/projected/27193a3e-6a72-4af3-881b-19706a3213f3-kube-api-access-sbhsm\") pod \"kuadrant-operator-catalog-5kgwk\" (UID: \"27193a3e-6a72-4af3-881b-19706a3213f3\") " pod="kuadrant-system/kuadrant-operator-catalog-5kgwk"
Jul 28 20:37:53.311198 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:53.311166    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbhsm\" (UniqueName: \"kubernetes.io/projected/27193a3e-6a72-4af3-881b-19706a3213f3-kube-api-access-sbhsm\") pod \"kuadrant-operator-catalog-5kgwk\" (UID: \"27193a3e-6a72-4af3-881b-19706a3213f3\") " pod="kuadrant-system/kuadrant-operator-catalog-5kgwk"
Jul 28 20:37:53.322680 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:53.322653    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbhsm\" (UniqueName: \"kubernetes.io/projected/27193a3e-6a72-4af3-881b-19706a3213f3-kube-api-access-sbhsm\") pod \"kuadrant-operator-catalog-5kgwk\" (UID: \"27193a3e-6a72-4af3-881b-19706a3213f3\") " pod="kuadrant-system/kuadrant-operator-catalog-5kgwk"
Jul 28 20:37:53.390072 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:53.390012    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk"
Jul 28 20:37:53.520575 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:53.520550    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kgwk"]
Jul 28 20:37:53.523059 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:37:53.523030    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27193a3e_6a72_4af3_881b_19706a3213f3.slice/crio-6b616c682ca22ef68a1c0f930fbfafa1f4de5b1859721b1e71d3a04cc366f283 WatchSource:0}: Error finding container 6b616c682ca22ef68a1c0f930fbfafa1f4de5b1859721b1e71d3a04cc366f283: Status 404 returned error can't find the container with id 6b616c682ca22ef68a1c0f930fbfafa1f4de5b1859721b1e71d3a04cc366f283
Jul 28 20:37:53.524552 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:53.524531    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:37:54.286553 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:54.286485    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk" event={"ID":"27193a3e-6a72-4af3-881b-19706a3213f3","Type":"ContainerStarted","Data":"6b616c682ca22ef68a1c0f930fbfafa1f4de5b1859721b1e71d3a04cc366f283"}
Jul 28 20:37:55.292644 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:55.292612    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk" event={"ID":"27193a3e-6a72-4af3-881b-19706a3213f3","Type":"ContainerStarted","Data":"eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684"}
Jul 28 20:37:55.307835 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:37:55.307788    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk" podStartSLOduration=0.692349801 podStartE2EDuration="2.307774296s" totalImagesPullingTime="1.615424495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:37:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:37:55.30658924 +0000 UTC m=+3056.487106299" watchObservedRunningTime="2026-07-28 20:37:55.307774296 +0000 UTC m=+3056.488291352"
Jul 28 20:38:03.390994 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:03.390902    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk"
Jul 28 20:38:03.390994 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:03.390948    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk"
Jul 28 20:38:03.412462 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:03.412405    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk"
Jul 28 20:38:04.368175 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:04.368147    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk"
Jul 28 20:38:04.412019 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:04.411984    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kgwk"]
Jul 28 20:38:06.356165 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:06.356127    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk" podUID="27193a3e-6a72-4af3-881b-19706a3213f3" containerName="registry-server" containerID="cri-o://eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684" gracePeriod=2
Jul 28 20:38:06.607480 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:06.607425    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk"
Jul 28 20:38:06.742195 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:06.742168    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbhsm\" (UniqueName: \"kubernetes.io/projected/27193a3e-6a72-4af3-881b-19706a3213f3-kube-api-access-sbhsm\") pod \"27193a3e-6a72-4af3-881b-19706a3213f3\" (UID: \"27193a3e-6a72-4af3-881b-19706a3213f3\") "
Jul 28 20:38:06.744126 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:06.744100    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27193a3e-6a72-4af3-881b-19706a3213f3-kube-api-access-sbhsm" (OuterVolumeSpecName: "kube-api-access-sbhsm") pod "27193a3e-6a72-4af3-881b-19706a3213f3" (UID: "27193a3e-6a72-4af3-881b-19706a3213f3"). InnerVolumeSpecName "kube-api-access-sbhsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:38:06.843235 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:06.843207    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbhsm\" (UniqueName: \"kubernetes.io/projected/27193a3e-6a72-4af3-881b-19706a3213f3-kube-api-access-sbhsm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:07.366515 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:07.366474    2569 generic.go:356] "Generic (PLEG): container finished" podID="27193a3e-6a72-4af3-881b-19706a3213f3" containerID="eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684" exitCode=0
Jul 28 20:38:07.366969 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:07.366636    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk"
Jul 28 20:38:07.371210 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:07.371177    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk" event={"ID":"27193a3e-6a72-4af3-881b-19706a3213f3","Type":"ContainerDied","Data":"eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684"}
Jul 28 20:38:07.371351 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:07.371226    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5kgwk" event={"ID":"27193a3e-6a72-4af3-881b-19706a3213f3","Type":"ContainerDied","Data":"6b616c682ca22ef68a1c0f930fbfafa1f4de5b1859721b1e71d3a04cc366f283"}
Jul 28 20:38:07.371351 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:07.371251    2569 scope.go:117] "RemoveContainer" containerID="eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684"
Jul 28 20:38:07.381469 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:07.381449    2569 scope.go:117] "RemoveContainer" containerID="eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684"
Jul 28 20:38:07.381707 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:38:07.381688    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684\": container with ID starting with eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684 not found: ID does not exist" containerID="eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684"
Jul 28 20:38:07.381750 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:07.381715    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684"} err="failed to get container status \"eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684\": rpc error: code = NotFound desc = could not find container \"eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684\": container with ID starting with eeae7403b3588c6c33094e127d5ccb148aed53bea8b8a90840970a1a72d6a684 not found: ID does not exist"
Jul 28 20:38:07.406218 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:07.406189    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kgwk"]
Jul 28 20:38:07.410321 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:07.410299    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kgwk"]
Jul 28 20:38:09.320317 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:09.320263    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"]
Jul 28 20:38:09.338861 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:09.338833    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:09.366549 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:09.366517    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27193a3e-6a72-4af3-881b-19706a3213f3" path="/var/lib/kubelet/pods/27193a3e-6a72-4af3-881b-19706a3213f3/volumes"
Jul 28 20:38:10.156623 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:10.156584    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"]
Jul 28 20:38:10.176364 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:10.176338    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:11.135465 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.135429    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-140563354683824"]
Jul 28 20:38:11.136099 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.136077    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27193a3e-6a72-4af3-881b-19706a3213f3" containerName="registry-server"
Jul 28 20:38:11.136145 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.136104    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="27193a3e-6a72-4af3-881b-19706a3213f3" containerName="registry-server"
Jul 28 20:38:11.136258 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.136244    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="27193a3e-6a72-4af3-881b-19706a3213f3" containerName="registry-server"
Jul 28 20:38:11.139647 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.139631    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354683824"
Jul 28 20:38:11.142108 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.142086    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lwr66\""
Jul 28 20:38:11.144713 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.144687    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354683824"]
Jul 28 20:38:11.183082 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.183058    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmhl6\" (UniqueName: \"kubernetes.io/projected/9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3-kube-api-access-qmhl6\") pod \"test-curl-7464-140563354683824\" (UID: \"9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3\") " pod="opendatahub/test-curl-7464-140563354683824"
Jul 28 20:38:11.283992 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.283965    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmhl6\" (UniqueName: \"kubernetes.io/projected/9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3-kube-api-access-qmhl6\") pod \"test-curl-7464-140563354683824\" (UID: \"9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3\") " pod="opendatahub/test-curl-7464-140563354683824"
Jul 28 20:38:11.291129 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.291106    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmhl6\" (UniqueName: \"kubernetes.io/projected/9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3-kube-api-access-qmhl6\") pod \"test-curl-7464-140563354683824\" (UID: \"9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3\") " pod="opendatahub/test-curl-7464-140563354683824"
Jul 28 20:38:11.451138 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.451111    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354683824"
Jul 28 20:38:11.588131 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:11.587676    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354683824"]
Jul 28 20:38:11.593473 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:38:11.593443    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c659cdd_b7b4_46ee_b9a8_df1bcc799cf3.slice/crio-3f54ca6ca307a79f2d0637aab0bc40d2a9a0bc28b48637838b7a8c0a4e72b0cd WatchSource:0}: Error finding container 3f54ca6ca307a79f2d0637aab0bc40d2a9a0bc28b48637838b7a8c0a4e72b0cd: Status 404 returned error can't find the container with id 3f54ca6ca307a79f2d0637aab0bc40d2a9a0bc28b48637838b7a8c0a4e72b0cd
Jul 28 20:38:12.393855 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:12.393828    2569 generic.go:356] "Generic (PLEG): container finished" podID="9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3" containerID="48735e4a82dfd5f775e748b527e33868d046f29e2e2fcfb7ef2849db392204ea" exitCode=0
Jul 28 20:38:12.394141 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:12.393909    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354683824" event={"ID":"9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3","Type":"ContainerDied","Data":"48735e4a82dfd5f775e748b527e33868d046f29e2e2fcfb7ef2849db392204ea"}
Jul 28 20:38:12.394141 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:12.393952    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354683824" event={"ID":"9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3","Type":"ContainerStarted","Data":"3f54ca6ca307a79f2d0637aab0bc40d2a9a0bc28b48637838b7a8c0a4e72b0cd"}
Jul 28 20:38:13.529814 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.529792    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354683824"
Jul 28 20:38:13.549189 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.549167    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-140563354683824_9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3/test-curl-7464-140563354683824/0.log"
Jul 28 20:38:13.572644 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.572566    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-140563354683824"]
Jul 28 20:38:13.574713 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.574691    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-140563354683824"]
Jul 28 20:38:13.604903 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.604880    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmhl6\" (UniqueName: \"kubernetes.io/projected/9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3-kube-api-access-qmhl6\") pod \"9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3\" (UID: \"9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3\") "
Jul 28 20:38:13.606882 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.606863    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3-kube-api-access-qmhl6" (OuterVolumeSpecName: "kube-api-access-qmhl6") pod "9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3" (UID: "9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3"). InnerVolumeSpecName "kube-api-access-qmhl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:38:13.698721 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.698699    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-140563354710704"]
Jul 28 20:38:13.699098 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.699086    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3" containerName="test-curl-7464-140563354683824"
Jul 28 20:38:13.699141 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.699099    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3" containerName="test-curl-7464-140563354683824"
Jul 28 20:38:13.699176 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.699169    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3" containerName="test-curl-7464-140563354683824"
Jul 28 20:38:13.701971 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.701957    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354710704"
Jul 28 20:38:13.705788 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.705760    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmhl6\" (UniqueName: \"kubernetes.io/projected/9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3-kube-api-access-qmhl6\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:13.706125 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.706103    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354710704"]
Jul 28 20:38:13.807051 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.807022    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6knv\" (UniqueName: \"kubernetes.io/projected/90669ca3-f0f5-4ca5-b5ce-01b04414ca1d-kube-api-access-x6knv\") pod \"test-curl-7464-140563354710704\" (UID: \"90669ca3-f0f5-4ca5-b5ce-01b04414ca1d\") " pod="opendatahub/test-curl-7464-140563354710704"
Jul 28 20:38:13.907844 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.907815    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6knv\" (UniqueName: \"kubernetes.io/projected/90669ca3-f0f5-4ca5-b5ce-01b04414ca1d-kube-api-access-x6knv\") pod \"test-curl-7464-140563354710704\" (UID: \"90669ca3-f0f5-4ca5-b5ce-01b04414ca1d\") " pod="opendatahub/test-curl-7464-140563354710704"
Jul 28 20:38:13.916666 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:13.916643    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6knv\" (UniqueName: \"kubernetes.io/projected/90669ca3-f0f5-4ca5-b5ce-01b04414ca1d-kube-api-access-x6knv\") pod \"test-curl-7464-140563354710704\" (UID: \"90669ca3-f0f5-4ca5-b5ce-01b04414ca1d\") " pod="opendatahub/test-curl-7464-140563354710704"
Jul 28 20:38:14.013075 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:14.013014    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354710704"
Jul 28 20:38:14.147404 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:14.147381    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354710704"]
Jul 28 20:38:14.149473 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:38:14.149451    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90669ca3_f0f5_4ca5_b5ce_01b04414ca1d.slice/crio-89136708e8020dc94421661d61800a6ff2c236a383bfe525023b7071e31fdbb1 WatchSource:0}: Error finding container 89136708e8020dc94421661d61800a6ff2c236a383bfe525023b7071e31fdbb1: Status 404 returned error can't find the container with id 89136708e8020dc94421661d61800a6ff2c236a383bfe525023b7071e31fdbb1
Jul 28 20:38:14.404744 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:14.404674    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354683824"
Jul 28 20:38:14.404880 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:14.404678    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f54ca6ca307a79f2d0637aab0bc40d2a9a0bc28b48637838b7a8c0a4e72b0cd"
Jul 28 20:38:14.405841 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:14.405818    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354710704" event={"ID":"90669ca3-f0f5-4ca5-b5ce-01b04414ca1d","Type":"ContainerStarted","Data":"89136708e8020dc94421661d61800a6ff2c236a383bfe525023b7071e31fdbb1"}
Jul 28 20:38:14.415001 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:38:14.414975    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7464-140563354683824\" is forbidden: User \"system:node:ip-10-0-134-138.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-138.ec2.internal' and this object" podUID="9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3" pod="opendatahub/test-curl-7464-140563354683824"
Jul 28 20:38:15.365624 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:15.365594    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3" path="/var/lib/kubelet/pods/9c659cdd-b7b4-46ee-b9a8-df1bcc799cf3/volumes"
Jul 28 20:38:15.413071 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:15.413039    2569 generic.go:356] "Generic (PLEG): container finished" podID="90669ca3-f0f5-4ca5-b5ce-01b04414ca1d" containerID="8c993c81a0af84a88286f3c5bd2316b6b34b90eaca7a537a855a5685a9a410cb" exitCode=0
Jul 28 20:38:15.413243 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:15.413140    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354710704" event={"ID":"90669ca3-f0f5-4ca5-b5ce-01b04414ca1d","Type":"ContainerDied","Data":"8c993c81a0af84a88286f3c5bd2316b6b34b90eaca7a537a855a5685a9a410cb"}
Jul 28 20:38:16.546536 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.546513    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354710704"
Jul 28 20:38:16.571547 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.571526    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-140563354710704_90669ca3-f0f5-4ca5-b5ce-01b04414ca1d/test-curl-7464-140563354710704/0.log"
Jul 28 20:38:16.603095 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.603075    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-140563354710704"]
Jul 28 20:38:16.605215 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.605193    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-140563354710704"]
Jul 28 20:38:16.635195 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.635170    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6knv\" (UniqueName: \"kubernetes.io/projected/90669ca3-f0f5-4ca5-b5ce-01b04414ca1d-kube-api-access-x6knv\") pod \"90669ca3-f0f5-4ca5-b5ce-01b04414ca1d\" (UID: \"90669ca3-f0f5-4ca5-b5ce-01b04414ca1d\") "
Jul 28 20:38:16.637108 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.637084    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90669ca3-f0f5-4ca5-b5ce-01b04414ca1d-kube-api-access-x6knv" (OuterVolumeSpecName: "kube-api-access-x6knv") pod "90669ca3-f0f5-4ca5-b5ce-01b04414ca1d" (UID: "90669ca3-f0f5-4ca5-b5ce-01b04414ca1d"). InnerVolumeSpecName "kube-api-access-x6knv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:38:16.736763 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.736727    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6knv\" (UniqueName: \"kubernetes.io/projected/90669ca3-f0f5-4ca5-b5ce-01b04414ca1d-kube-api-access-x6knv\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:16.797307 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.797269    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-140563354851344"]
Jul 28 20:38:16.797698 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.797686    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90669ca3-f0f5-4ca5-b5ce-01b04414ca1d" containerName="test-curl-7464-140563354710704"
Jul 28 20:38:16.797741 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.797700    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="90669ca3-f0f5-4ca5-b5ce-01b04414ca1d" containerName="test-curl-7464-140563354710704"
Jul 28 20:38:16.797789 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.797778    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="90669ca3-f0f5-4ca5-b5ce-01b04414ca1d" containerName="test-curl-7464-140563354710704"
Jul 28 20:38:16.800908 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.800894    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:16.810545 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.808753    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354851344"]
Jul 28 20:38:16.837810 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.837786    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78rng\" (UniqueName: \"kubernetes.io/projected/193f14bd-29b3-4940-907f-61cafd770383-kube-api-access-78rng\") pod \"test-curl-7464-140563354851344\" (UID: \"193f14bd-29b3-4940-907f-61cafd770383\") " pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:16.938600 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.938578    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78rng\" (UniqueName: \"kubernetes.io/projected/193f14bd-29b3-4940-907f-61cafd770383-kube-api-access-78rng\") pod \"test-curl-7464-140563354851344\" (UID: \"193f14bd-29b3-4940-907f-61cafd770383\") " pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:16.945635 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:16.945612    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78rng\" (UniqueName: \"kubernetes.io/projected/193f14bd-29b3-4940-907f-61cafd770383-kube-api-access-78rng\") pod \"test-curl-7464-140563354851344\" (UID: \"193f14bd-29b3-4940-907f-61cafd770383\") " pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:17.120314 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:17.120232    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:17.251715 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:17.251692    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354851344"]
Jul 28 20:38:17.253318 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:38:17.253294    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod193f14bd_29b3_4940_907f_61cafd770383.slice/crio-6d1496005acf4c7d7253906c4c5ca1eb023fadfd240d40b19248e21334efaeb0 WatchSource:0}: Error finding container 6d1496005acf4c7d7253906c4c5ca1eb023fadfd240d40b19248e21334efaeb0: Status 404 returned error can't find the container with id 6d1496005acf4c7d7253906c4c5ca1eb023fadfd240d40b19248e21334efaeb0
Jul 28 20:38:17.365904 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:17.365875    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90669ca3-f0f5-4ca5-b5ce-01b04414ca1d" path="/var/lib/kubelet/pods/90669ca3-f0f5-4ca5-b5ce-01b04414ca1d/volumes"
Jul 28 20:38:17.426311 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:17.426292    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354710704"
Jul 28 20:38:17.426433 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:17.426332    2569 scope.go:117] "RemoveContainer" containerID="8c993c81a0af84a88286f3c5bd2316b6b34b90eaca7a537a855a5685a9a410cb"
Jul 28 20:38:17.427700 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:17.427679    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354851344" event={"ID":"193f14bd-29b3-4940-907f-61cafd770383","Type":"ContainerStarted","Data":"6d1496005acf4c7d7253906c4c5ca1eb023fadfd240d40b19248e21334efaeb0"}
Jul 28 20:38:18.435767 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:18.435728    2569 generic.go:356] "Generic (PLEG): container finished" podID="193f14bd-29b3-4940-907f-61cafd770383" containerID="dfbcfb9bfab2993a9f918ed6a393abe4593b10c522161fb7780e3fb793500eb0" exitCode=0
Jul 28 20:38:18.436128 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:18.435778    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354851344" event={"ID":"193f14bd-29b3-4940-907f-61cafd770383","Type":"ContainerDied","Data":"dfbcfb9bfab2993a9f918ed6a393abe4593b10c522161fb7780e3fb793500eb0"}
Jul 28 20:38:19.575404 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.575374    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:19.597629 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.597607    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-140563354851344_193f14bd-29b3-4940-907f-61cafd770383/test-curl-7464-140563354851344/0.log"
Jul 28 20:38:19.622597 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.622574    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-140563354851344"]
Jul 28 20:38:19.626583 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.626561    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-140563354851344"]
Jul 28 20:38:19.661485 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.661463    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78rng\" (UniqueName: \"kubernetes.io/projected/193f14bd-29b3-4940-907f-61cafd770383-kube-api-access-78rng\") pod \"193f14bd-29b3-4940-907f-61cafd770383\" (UID: \"193f14bd-29b3-4940-907f-61cafd770383\") "
Jul 28 20:38:19.663386 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.663361    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/193f14bd-29b3-4940-907f-61cafd770383-kube-api-access-78rng" (OuterVolumeSpecName: "kube-api-access-78rng") pod "193f14bd-29b3-4940-907f-61cafd770383" (UID: "193f14bd-29b3-4940-907f-61cafd770383"). InnerVolumeSpecName "kube-api-access-78rng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:38:19.752506 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.752451    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-140563354851344"]
Jul 28 20:38:19.752844 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.752831    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="193f14bd-29b3-4940-907f-61cafd770383" containerName="test-curl-7464-140563354851344"
Jul 28 20:38:19.752888 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.752845    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="193f14bd-29b3-4940-907f-61cafd770383" containerName="test-curl-7464-140563354851344"
Jul 28 20:38:19.752924 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.752912    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="193f14bd-29b3-4940-907f-61cafd770383" containerName="test-curl-7464-140563354851344"
Jul 28 20:38:19.755805 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.755788    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:19.762495 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.762343    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78rng\" (UniqueName: \"kubernetes.io/projected/193f14bd-29b3-4940-907f-61cafd770383-kube-api-access-78rng\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:19.763482 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.763462    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354851344"]
Jul 28 20:38:19.863655 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.863630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mddvl\" (UniqueName: \"kubernetes.io/projected/e03827ab-792d-4db8-b8c2-d4da30d42086-kube-api-access-mddvl\") pod \"test-curl-7464-140563354851344\" (UID: \"e03827ab-792d-4db8-b8c2-d4da30d42086\") " pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:19.965169 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.965122    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mddvl\" (UniqueName: \"kubernetes.io/projected/e03827ab-792d-4db8-b8c2-d4da30d42086-kube-api-access-mddvl\") pod \"test-curl-7464-140563354851344\" (UID: \"e03827ab-792d-4db8-b8c2-d4da30d42086\") " pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:19.973857 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:19.973834    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mddvl\" (UniqueName: \"kubernetes.io/projected/e03827ab-792d-4db8-b8c2-d4da30d42086-kube-api-access-mddvl\") pod \"test-curl-7464-140563354851344\" (UID: \"e03827ab-792d-4db8-b8c2-d4da30d42086\") " pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:20.065198 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:20.065146    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:20.193080 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:20.193053    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354851344"]
Jul 28 20:38:20.195211 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:38:20.195181    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode03827ab_792d_4db8_b8c2_d4da30d42086.slice/crio-b776919667b9f1aab4879d8e653efff5f8ee2dc8669c586e55732ae655d70a3e WatchSource:0}: Error finding container b776919667b9f1aab4879d8e653efff5f8ee2dc8669c586e55732ae655d70a3e: Status 404 returned error can't find the container with id b776919667b9f1aab4879d8e653efff5f8ee2dc8669c586e55732ae655d70a3e
Jul 28 20:38:20.447876 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:20.447842    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354851344" event={"ID":"e03827ab-792d-4db8-b8c2-d4da30d42086","Type":"ContainerStarted","Data":"b776919667b9f1aab4879d8e653efff5f8ee2dc8669c586e55732ae655d70a3e"}
Jul 28 20:38:20.448975 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:20.448956    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d1496005acf4c7d7253906c4c5ca1eb023fadfd240d40b19248e21334efaeb0"
Jul 28 20:38:20.449071 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:20.448988    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:20.460872 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:20.460828    2569 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7464-140563354851344" oldPodUID="193f14bd-29b3-4940-907f-61cafd770383" podUID="e03827ab-792d-4db8-b8c2-d4da30d42086"
Jul 28 20:38:21.366826 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:21.366792    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="193f14bd-29b3-4940-907f-61cafd770383" path="/var/lib/kubelet/pods/193f14bd-29b3-4940-907f-61cafd770383/volumes"
Jul 28 20:38:21.455271 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:21.455239    2569 generic.go:356] "Generic (PLEG): container finished" podID="e03827ab-792d-4db8-b8c2-d4da30d42086" containerID="c941eb6d57e7454aad99c9082654cf10fed117e84e1e9b770ee0d1567c0e2dd7" exitCode=0
Jul 28 20:38:21.455407 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:21.455325    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354851344" event={"ID":"e03827ab-792d-4db8-b8c2-d4da30d42086","Type":"ContainerDied","Data":"c941eb6d57e7454aad99c9082654cf10fed117e84e1e9b770ee0d1567c0e2dd7"}
Jul 28 20:38:22.586802 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.586781    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:22.609854 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.609833    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-140563354851344_e03827ab-792d-4db8-b8c2-d4da30d42086/test-curl-7464-140563354851344/0.log"
Jul 28 20:38:22.636143 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.636119    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-140563354851344"]
Jul 28 20:38:22.638055 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.638037    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-140563354851344"]
Jul 28 20:38:22.689327 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.689304    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mddvl\" (UniqueName: \"kubernetes.io/projected/e03827ab-792d-4db8-b8c2-d4da30d42086-kube-api-access-mddvl\") pod \"e03827ab-792d-4db8-b8c2-d4da30d42086\" (UID: \"e03827ab-792d-4db8-b8c2-d4da30d42086\") "
Jul 28 20:38:22.691333 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.691305    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e03827ab-792d-4db8-b8c2-d4da30d42086-kube-api-access-mddvl" (OuterVolumeSpecName: "kube-api-access-mddvl") pod "e03827ab-792d-4db8-b8c2-d4da30d42086" (UID: "e03827ab-792d-4db8-b8c2-d4da30d42086"). InnerVolumeSpecName "kube-api-access-mddvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:38:22.764220 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.764163    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-140563354852352"]
Jul 28 20:38:22.764626 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.764609    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e03827ab-792d-4db8-b8c2-d4da30d42086" containerName="test-curl-7464-140563354851344"
Jul 28 20:38:22.764709 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.764628    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e03827ab-792d-4db8-b8c2-d4da30d42086" containerName="test-curl-7464-140563354851344"
Jul 28 20:38:22.764771 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.764763    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e03827ab-792d-4db8-b8c2-d4da30d42086" containerName="test-curl-7464-140563354851344"
Jul 28 20:38:22.767820 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.767801    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:22.774447 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.774422    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354852352"]
Jul 28 20:38:22.791080 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.791052    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mddvl\" (UniqueName: \"kubernetes.io/projected/e03827ab-792d-4db8-b8c2-d4da30d42086-kube-api-access-mddvl\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:22.891730 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.891706    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x67l6\" (UniqueName: \"kubernetes.io/projected/4f1d03bf-5d8b-4a1f-8303-d978e87cde57-kube-api-access-x67l6\") pod \"test-curl-7464-140563354852352\" (UID: \"4f1d03bf-5d8b-4a1f-8303-d978e87cde57\") " pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:22.992315 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:22.992272    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x67l6\" (UniqueName: \"kubernetes.io/projected/4f1d03bf-5d8b-4a1f-8303-d978e87cde57-kube-api-access-x67l6\") pod \"test-curl-7464-140563354852352\" (UID: \"4f1d03bf-5d8b-4a1f-8303-d978e87cde57\") " pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:23.001128 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:23.001106    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x67l6\" (UniqueName: \"kubernetes.io/projected/4f1d03bf-5d8b-4a1f-8303-d978e87cde57-kube-api-access-x67l6\") pod \"test-curl-7464-140563354852352\" (UID: \"4f1d03bf-5d8b-4a1f-8303-d978e87cde57\") " pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:23.078300 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:23.078227    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:23.208361 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:23.208330    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354852352"]
Jul 28 20:38:23.211373 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:38:23.211334    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f1d03bf_5d8b_4a1f_8303_d978e87cde57.slice/crio-3881053587ff210a5780746c09cceab361bc3492b5ddc383810d6a918d6716c7 WatchSource:0}: Error finding container 3881053587ff210a5780746c09cceab361bc3492b5ddc383810d6a918d6716c7: Status 404 returned error can't find the container with id 3881053587ff210a5780746c09cceab361bc3492b5ddc383810d6a918d6716c7
Jul 28 20:38:23.366260 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:23.366196    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e03827ab-792d-4db8-b8c2-d4da30d42086" path="/var/lib/kubelet/pods/e03827ab-792d-4db8-b8c2-d4da30d42086/volumes"
Jul 28 20:38:23.469048 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:23.469024    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354851344"
Jul 28 20:38:23.469195 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:23.469052    2569 scope.go:117] "RemoveContainer" containerID="c941eb6d57e7454aad99c9082654cf10fed117e84e1e9b770ee0d1567c0e2dd7"
Jul 28 20:38:23.470359 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:23.470337    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354852352" event={"ID":"4f1d03bf-5d8b-4a1f-8303-d978e87cde57","Type":"ContainerStarted","Data":"3881053587ff210a5780746c09cceab361bc3492b5ddc383810d6a918d6716c7"}
Jul 28 20:38:24.339725 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.339682    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4" podUID="cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" containerName="main" containerID="cri-o://82a93acd53b1be3506b0e253c71d5e511b54669416050f4b7d39aaf9d14a33cb" gracePeriod=45
Jul 28 20:38:24.348795 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.348768    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:24.478813 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.478781    2569 generic.go:356] "Generic (PLEG): container finished" podID="4f1d03bf-5d8b-4a1f-8303-d978e87cde57" containerID="6806228091cdc1dce04190c32c7319dc8a743052642e6ef50c1db7aed3b8baea" exitCode=0
Jul 28 20:38:24.478955 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.478857    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354852352" event={"ID":"4f1d03bf-5d8b-4a1f-8303-d978e87cde57","Type":"ContainerDied","Data":"6806228091cdc1dce04190c32c7319dc8a743052642e6ef50c1db7aed3b8baea"}
Jul 28 20:38:24.480563 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.480542    2569 generic.go:356] "Generic (PLEG): container finished" podID="cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" containerID="82a93acd53b1be3506b0e253c71d5e511b54669416050f4b7d39aaf9d14a33cb" exitCode=0
Jul 28 20:38:24.480674 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.480572    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4" event={"ID":"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9","Type":"ContainerDied","Data":"82a93acd53b1be3506b0e253c71d5e511b54669416050f4b7d39aaf9d14a33cb"}
Jul 28 20:38:24.591070 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.591019    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:38:24.707359 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.707334    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-dshm\") pod \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") "
Jul 28 20:38:24.707513 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.707369    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmzpc\" (UniqueName: \"kubernetes.io/projected/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-kube-api-access-kmzpc\") pod \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") "
Jul 28 20:38:24.707513 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.707441    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tmp-dir\") pod \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") "
Jul 28 20:38:24.707513 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.707485    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-model-cache\") pod \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") "
Jul 28 20:38:24.707513 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.707509    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tls-certs\") pod \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") "
Jul 28 20:38:24.707679 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.707538    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-home\") pod \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\" (UID: \"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9\") "
Jul 28 20:38:24.707819 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.707784    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" (UID: "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:38:24.707896 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.707794    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-model-cache" (OuterVolumeSpecName: "model-cache") pod "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" (UID: "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:38:24.707945 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.707889    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-home" (OuterVolumeSpecName: "home") pod "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" (UID: "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:38:24.709500 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.709470    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-dshm" (OuterVolumeSpecName: "dshm") pod "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" (UID: "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:38:24.709612 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.709534    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" (UID: "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:38:24.709612 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.709548    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-kube-api-access-kmzpc" (OuterVolumeSpecName: "kube-api-access-kmzpc") pod "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" (UID: "cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9"). InnerVolumeSpecName "kube-api-access-kmzpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:38:24.808363 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.808337    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:24.808363 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.808359    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:24.808499 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.808368    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:24.808499 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.808375    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:24.808499 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.808382    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:24.808499 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:24.808389    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmzpc\" (UniqueName: \"kubernetes.io/projected/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9-kube-api-access-kmzpc\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:25.176121 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.176068    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2" podUID="5801b420-8258-40fa-a796-60e10d46f9db" containerName="main" containerID="cri-o://b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56" gracePeriod=45
Jul 28 20:38:25.414849 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.414823    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:38:25.486858 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.486784    2569 generic.go:356] "Generic (PLEG): container finished" podID="5801b420-8258-40fa-a796-60e10d46f9db" containerID="b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56" exitCode=0
Jul 28 20:38:25.486858 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.486831    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"
Jul 28 20:38:25.487036 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.486872    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2" event={"ID":"5801b420-8258-40fa-a796-60e10d46f9db","Type":"ContainerDied","Data":"b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56"}
Jul 28 20:38:25.487036 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.486917    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2" event={"ID":"5801b420-8258-40fa-a796-60e10d46f9db","Type":"ContainerDied","Data":"cb10387e33e68c3d58a71b46cdbc5c153d18528f8161ecb9ff17e216b4a76e6e"}
Jul 28 20:38:25.487036 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.486938    2569 scope.go:117] "RemoveContainer" containerID="b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56"
Jul 28 20:38:25.488342 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.488310    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4" event={"ID":"cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9","Type":"ContainerDied","Data":"9a9bede2017dbcdb43ba730c12e5134113675173dd74cc0b5fc21cdd1cf46528"}
Jul 28 20:38:25.488342 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.488336    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"
Jul 28 20:38:25.496009 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.495991    2569 scope.go:117] "RemoveContainer" containerID="b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56"
Jul 28 20:38:25.496257 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:38:25.496236    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56\": container with ID starting with b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56 not found: ID does not exist" containerID="b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56"
Jul 28 20:38:25.496336 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.496268    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56"} err="failed to get container status \"b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56\": rpc error: code = NotFound desc = could not find container \"b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56\": container with ID starting with b50839adafad8f7c3f21151e5d094221d10dcbf58b8e1ae810f2560ee2505a56 not found: ID does not exist"
Jul 28 20:38:25.496336 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.496312    2569 scope.go:117] "RemoveContainer" containerID="82a93acd53b1be3506b0e253c71d5e511b54669416050f4b7d39aaf9d14a33cb"
Jul 28 20:38:25.508225 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.508186    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"]
Jul 28 20:38:25.510355 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.510268    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ed54db/rate-test-model-ed54db-kserve-86488b888c-zhhf4"]
Jul 28 20:38:25.514548 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.514526    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqvmr\" (UniqueName: \"kubernetes.io/projected/5801b420-8258-40fa-a796-60e10d46f9db-kube-api-access-jqvmr\") pod \"5801b420-8258-40fa-a796-60e10d46f9db\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") "
Jul 28 20:38:25.514653 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.514587    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-model-cache\") pod \"5801b420-8258-40fa-a796-60e10d46f9db\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") "
Jul 28 20:38:25.514653 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.514627    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5801b420-8258-40fa-a796-60e10d46f9db-tls-certs\") pod \"5801b420-8258-40fa-a796-60e10d46f9db\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") "
Jul 28 20:38:25.514760 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.514691    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-tmp-dir\") pod \"5801b420-8258-40fa-a796-60e10d46f9db\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") "
Jul 28 20:38:25.514760 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.514738    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-home\") pod \"5801b420-8258-40fa-a796-60e10d46f9db\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") "
Jul 28 20:38:25.514868 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.514777    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-dshm\") pod \"5801b420-8258-40fa-a796-60e10d46f9db\" (UID: \"5801b420-8258-40fa-a796-60e10d46f9db\") "
Jul 28 20:38:25.514868 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.514835    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-model-cache" (OuterVolumeSpecName: "model-cache") pod "5801b420-8258-40fa-a796-60e10d46f9db" (UID: "5801b420-8258-40fa-a796-60e10d46f9db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:38:25.515018 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.515004    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5801b420-8258-40fa-a796-60e10d46f9db" (UID: "5801b420-8258-40fa-a796-60e10d46f9db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:38:25.515119 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.515097    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-home" (OuterVolumeSpecName: "home") pod "5801b420-8258-40fa-a796-60e10d46f9db" (UID: "5801b420-8258-40fa-a796-60e10d46f9db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:38:25.515197 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.515144    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:25.515197 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.515161    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:25.516798 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.516766    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5801b420-8258-40fa-a796-60e10d46f9db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5801b420-8258-40fa-a796-60e10d46f9db" (UID: "5801b420-8258-40fa-a796-60e10d46f9db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:38:25.517195 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.517170    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-dshm" (OuterVolumeSpecName: "dshm") pod "5801b420-8258-40fa-a796-60e10d46f9db" (UID: "5801b420-8258-40fa-a796-60e10d46f9db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:38:25.517195 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.517181    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5801b420-8258-40fa-a796-60e10d46f9db-kube-api-access-jqvmr" (OuterVolumeSpecName: "kube-api-access-jqvmr") pod "5801b420-8258-40fa-a796-60e10d46f9db" (UID: "5801b420-8258-40fa-a796-60e10d46f9db"). InnerVolumeSpecName "kube-api-access-jqvmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:38:25.615913 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.615886    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:25.615913 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.615911    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5801b420-8258-40fa-a796-60e10d46f9db-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:25.616089 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.615920    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqvmr\" (UniqueName: \"kubernetes.io/projected/5801b420-8258-40fa-a796-60e10d46f9db-kube-api-access-jqvmr\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:25.616089 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.615929    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5801b420-8258-40fa-a796-60e10d46f9db-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:25.628649 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.628632    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:25.648008 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.647976    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-140563354852352_4f1d03bf-5d8b-4a1f-8303-d978e87cde57/test-curl-7464-140563354852352/0.log"
Jul 28 20:38:25.673243 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.673220    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-140563354852352"]
Jul 28 20:38:25.677361 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.677340    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-140563354852352"]
Jul 28 20:38:25.716559 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.716536    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x67l6\" (UniqueName: \"kubernetes.io/projected/4f1d03bf-5d8b-4a1f-8303-d978e87cde57-kube-api-access-x67l6\") pod \"4f1d03bf-5d8b-4a1f-8303-d978e87cde57\" (UID: \"4f1d03bf-5d8b-4a1f-8303-d978e87cde57\") "
Jul 28 20:38:25.718578 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.718554    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f1d03bf-5d8b-4a1f-8303-d978e87cde57-kube-api-access-x67l6" (OuterVolumeSpecName: "kube-api-access-x67l6") pod "4f1d03bf-5d8b-4a1f-8303-d978e87cde57" (UID: "4f1d03bf-5d8b-4a1f-8303-d978e87cde57"). InnerVolumeSpecName "kube-api-access-x67l6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:38:25.802471 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.802452    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-140563354852352"]
Jul 28 20:38:25.802852 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.802841    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5801b420-8258-40fa-a796-60e10d46f9db" containerName="main"
Jul 28 20:38:25.802897 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.802853    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5801b420-8258-40fa-a796-60e10d46f9db" containerName="main"
Jul 28 20:38:25.802897 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.802865    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" containerName="main"
Jul 28 20:38:25.802897 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.802870    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" containerName="main"
Jul 28 20:38:25.802897 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.802881    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f1d03bf-5d8b-4a1f-8303-d978e87cde57" containerName="test-curl-7464-140563354852352"
Jul 28 20:38:25.802897 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.802887    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f1d03bf-5d8b-4a1f-8303-d978e87cde57" containerName="test-curl-7464-140563354852352"
Jul 28 20:38:25.803049 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.802938    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5801b420-8258-40fa-a796-60e10d46f9db" containerName="main"
Jul 28 20:38:25.803049 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.802949    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" containerName="main"
Jul 28 20:38:25.803049 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.802958    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f1d03bf-5d8b-4a1f-8303-d978e87cde57" containerName="test-curl-7464-140563354852352"
Jul 28 20:38:25.806071 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.806052    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:25.807322 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.807304    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"]
Jul 28 20:38:25.814694 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.813003    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354852352"]
Jul 28 20:38:25.815437 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.815416    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1231ba/rate-test-model-1231ba-kserve-67cbbb88c-6rth2"]
Jul 28 20:38:25.817808 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.817782    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x67l6\" (UniqueName: \"kubernetes.io/projected/4f1d03bf-5d8b-4a1f-8303-d978e87cde57-kube-api-access-x67l6\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:25.918699 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:25.918672    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmthn\" (UniqueName: \"kubernetes.io/projected/7a3bc822-c05c-43ad-b700-480aa95c1df1-kube-api-access-fmthn\") pod \"test-curl-7464-140563354852352\" (UID: \"7a3bc822-c05c-43ad-b700-480aa95c1df1\") " pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:26.019775 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:26.019744    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmthn\" (UniqueName: \"kubernetes.io/projected/7a3bc822-c05c-43ad-b700-480aa95c1df1-kube-api-access-fmthn\") pod \"test-curl-7464-140563354852352\" (UID: \"7a3bc822-c05c-43ad-b700-480aa95c1df1\") " pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:26.029214 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:26.029187    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmthn\" (UniqueName: \"kubernetes.io/projected/7a3bc822-c05c-43ad-b700-480aa95c1df1-kube-api-access-fmthn\") pod \"test-curl-7464-140563354852352\" (UID: \"7a3bc822-c05c-43ad-b700-480aa95c1df1\") " pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:26.117067 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:26.117013    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:26.245732 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:26.245707    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-140563354852352"]
Jul 28 20:38:26.248555 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:38:26.248528    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a3bc822_c05c_43ad_b700_480aa95c1df1.slice/crio-0f4730f80c643811751e8f7197da090166d248a5fc208289936a9897bb104fe5 WatchSource:0}: Error finding container 0f4730f80c643811751e8f7197da090166d248a5fc208289936a9897bb104fe5: Status 404 returned error can't find the container with id 0f4730f80c643811751e8f7197da090166d248a5fc208289936a9897bb104fe5
Jul 28 20:38:26.495642 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:26.495619    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:26.495642 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:26.495637    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3881053587ff210a5780746c09cceab361bc3492b5ddc383810d6a918d6716c7"
Jul 28 20:38:26.496795 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:26.496769    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354852352" event={"ID":"7a3bc822-c05c-43ad-b700-480aa95c1df1","Type":"ContainerStarted","Data":"0f4730f80c643811751e8f7197da090166d248a5fc208289936a9897bb104fe5"}
Jul 28 20:38:26.507160 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:26.507134    2569 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7464-140563354852352" oldPodUID="4f1d03bf-5d8b-4a1f-8303-d978e87cde57" podUID="7a3bc822-c05c-43ad-b700-480aa95c1df1"
Jul 28 20:38:27.365693 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:27.365664    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f1d03bf-5d8b-4a1f-8303-d978e87cde57" path="/var/lib/kubelet/pods/4f1d03bf-5d8b-4a1f-8303-d978e87cde57/volumes"
Jul 28 20:38:27.366047 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:27.366034    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5801b420-8258-40fa-a796-60e10d46f9db" path="/var/lib/kubelet/pods/5801b420-8258-40fa-a796-60e10d46f9db/volumes"
Jul 28 20:38:27.366411 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:27.366398    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9" path="/var/lib/kubelet/pods/cf83c241-c8a4-4d5b-8ba0-e191a4e98fb9/volumes"
Jul 28 20:38:27.511675 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:27.511638    2569 generic.go:356] "Generic (PLEG): container finished" podID="7a3bc822-c05c-43ad-b700-480aa95c1df1" containerID="0a1a799e8daf16b78c8de56300b1b87f2acf01902f2223b7336afe5ee8314fd9" exitCode=0
Jul 28 20:38:27.512062 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:27.511717    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-140563354852352" event={"ID":"7a3bc822-c05c-43ad-b700-480aa95c1df1","Type":"ContainerDied","Data":"0a1a799e8daf16b78c8de56300b1b87f2acf01902f2223b7336afe5ee8314fd9"}
Jul 28 20:38:28.652124 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:28.652098    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:28.674725 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:28.674697    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-140563354852352_7a3bc822-c05c-43ad-b700-480aa95c1df1/test-curl-7464-140563354852352/0.log"
Jul 28 20:38:28.701959 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:28.701931    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-140563354852352"]
Jul 28 20:38:28.709397 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:28.709375    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-140563354852352"]
Jul 28 20:38:28.742911 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:28.742892    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmthn\" (UniqueName: \"kubernetes.io/projected/7a3bc822-c05c-43ad-b700-480aa95c1df1-kube-api-access-fmthn\") pod \"7a3bc822-c05c-43ad-b700-480aa95c1df1\" (UID: \"7a3bc822-c05c-43ad-b700-480aa95c1df1\") "
Jul 28 20:38:28.744823 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:28.744797    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a3bc822-c05c-43ad-b700-480aa95c1df1-kube-api-access-fmthn" (OuterVolumeSpecName: "kube-api-access-fmthn") pod "7a3bc822-c05c-43ad-b700-480aa95c1df1" (UID: "7a3bc822-c05c-43ad-b700-480aa95c1df1"). InnerVolumeSpecName "kube-api-access-fmthn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:38:28.843768 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:28.843706    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmthn\" (UniqueName: \"kubernetes.io/projected/7a3bc822-c05c-43ad-b700-480aa95c1df1-kube-api-access-fmthn\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:38:29.237796 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:29.237768    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:29.365896 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:29.365866    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a3bc822-c05c-43ad-b700-480aa95c1df1" path="/var/lib/kubelet/pods/7a3bc822-c05c-43ad-b700-480aa95c1df1/volumes"
Jul 28 20:38:29.526732 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:29.526654    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-140563354852352"
Jul 28 20:38:29.526732 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:29.526680    2569 scope.go:117] "RemoveContainer" containerID="0a1a799e8daf16b78c8de56300b1b87f2acf01902f2223b7336afe5ee8314fd9"
Jul 28 20:38:29.568021 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:29.567990    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:29.801432 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:29.801355    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:32.879499 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:32.879462    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:33.000468 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:33.000435    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:33.104839 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:33.104808    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:33.308801 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:33.308773    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:33.698851 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:33.698817    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:33.812822 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:33.812792    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:57.958036 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:57.958004    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:58.709405 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:58.709375    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:59.088346 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:59.088255    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:59.877871 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:38:59.877817    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:01.977263 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:01.977225    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:02.079025 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:02.078997    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:02.159468 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:02.159440    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:02.310607 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:02.310531    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:02.409392 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:02.409363    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:02.659223 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:02.659146    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:02.759139 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:02.759108    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:27.016015 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:27.015933    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:42.757576 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.757543    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"]
Jul 28 20:39:42.758064 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.757931    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3bc822-c05c-43ad-b700-480aa95c1df1" containerName="test-curl-7464-140563354852352"
Jul 28 20:39:42.758064 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.757944    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3bc822-c05c-43ad-b700-480aa95c1df1" containerName="test-curl-7464-140563354852352"
Jul 28 20:39:42.758064 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.758027    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a3bc822-c05c-43ad-b700-480aa95c1df1" containerName="test-curl-7464-140563354852352"
Jul 28 20:39:42.761108 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.761086    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.766867 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.766849    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-92ba10\"/\"default-dockercfg-9krhr\""
Jul 28 20:39:42.768202 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.768186    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-92ba10\"/\"kube-root-ca.crt\""
Jul 28 20:39:42.768649 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.768632    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-92ba10\"/\"openshift-service-ca.crt\""
Jul 28 20:39:42.769682 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.769668    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-92ba10\"/\"ipp-route-92ba10-kserve-self-signed-certs\""
Jul 28 20:39:42.811177 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.811151    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"]
Jul 28 20:39:42.872614 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.872588    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-dshm\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.872709 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.872620    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a38975b4-d3ca-45a4-9336-16f4fd075199-tls-certs\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.872755 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.872711    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-home\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.872791 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.872754    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-tmp-dir\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.872826 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.872815    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-model-cache\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.872864 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.872847    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpdds\" (UniqueName: \"kubernetes.io/projected/a38975b4-d3ca-45a4-9336-16f4fd075199-kube-api-access-xpdds\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.907063 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.907038    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:42.973833 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.973808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-model-cache\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.973952 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.973845    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpdds\" (UniqueName: \"kubernetes.io/projected/a38975b4-d3ca-45a4-9336-16f4fd075199-kube-api-access-xpdds\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.973952 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.973870    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-dshm\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.973952 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.973889    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a38975b4-d3ca-45a4-9336-16f4fd075199-tls-certs\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.973952 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.973933    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-home\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.973952 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.973951    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-tmp-dir\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.974396 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.974373    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-tmp-dir\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.974498 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.974380    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-home\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.974498 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.974416    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-model-cache\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.976087 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.976070    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-dshm\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.976471 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.976453    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a38975b4-d3ca-45a4-9336-16f4fd075199-tls-certs\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:42.987376 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:42.987349    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpdds\" (UniqueName: \"kubernetes.io/projected/a38975b4-d3ca-45a4-9336-16f4fd075199-kube-api-access-xpdds\") pod \"ipp-route-92ba10-kserve-959bf8695-v68zc\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") " pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:43.070832 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:43.070779    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:43.210745 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:43.210720    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"]
Jul 28 20:39:43.213862 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:39:43.213838    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda38975b4_d3ca_45a4_9336_16f4fd075199.slice/crio-3f3f89d04c61037372578cde6811764b3f98471e93dbe0c328aac52a0b6c86d5 WatchSource:0}: Error finding container 3f3f89d04c61037372578cde6811764b3f98471e93dbe0c328aac52a0b6c86d5: Status 404 returned error can't find the container with id 3f3f89d04c61037372578cde6811764b3f98471e93dbe0c328aac52a0b6c86d5
Jul 28 20:39:43.907618 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:43.907582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc" event={"ID":"a38975b4-d3ca-45a4-9336-16f4fd075199","Type":"ContainerStarted","Data":"f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44"}
Jul 28 20:39:43.907618 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:43.907622    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc" event={"ID":"a38975b4-d3ca-45a4-9336-16f4fd075199","Type":"ContainerStarted","Data":"3f3f89d04c61037372578cde6811764b3f98471e93dbe0c328aac52a0b6c86d5"}
Jul 28 20:39:43.928954 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:43.928911    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc" podStartSLOduration=1.9288935299999999 podStartE2EDuration="1.92889353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:39:43.925131997 +0000 UTC m=+3165.105649050" watchObservedRunningTime="2026-07-28 20:39:43.92889353 +0000 UTC m=+3165.109410590"
Jul 28 20:39:53.071199 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:53.071163    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:53.071669 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:53.071212    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:53.083331 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:53.083311    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:39:53.970085 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:39:53.970053    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:40:27.154094 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:40:27.154061    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:27.589231 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:40:27.589195    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:28.378997 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:40:28.378964    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:30.691527 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:40:30.691488    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:30.799510 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:40:30.799470    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:30.995917 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:40:30.995844    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:31.426828 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:40:31.426798    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:31.638935 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:40:31.638906    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:31.882001 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:40:31.881933    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:59.711826 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:40:59.711749    2569 scope.go:117] "RemoveContainer" containerID="38ed7cd4b06406075bdfa6e20f381ac18f79c9076c628cfe0493447ead09ea3c"
Jul 28 20:41:00.864267 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:00.864231    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:01.572188 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:01.572147    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"]
Jul 28 20:41:01.577074 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:01.577050    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"
Jul 28 20:41:01.579158 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:01.579135    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:41:01.579265 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:01.579176    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:41:01.592926 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:01.592628    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"]
Jul 28 20:41:01.639957 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:01.639922    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d82f463-3a4e-4344-b9cb-a4beaf44a0b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4\" (UID: \"2d82f463-3a4e-4344-b9cb-a4beaf44a0b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"
Jul 28 20:41:01.740985 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:01.740949    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d82f463-3a4e-4344-b9cb-a4beaf44a0b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4\" (UID: \"2d82f463-3a4e-4344-b9cb-a4beaf44a0b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"
Jul 28 20:41:01.741688 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:01.741668    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d82f463-3a4e-4344-b9cb-a4beaf44a0b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4\" (UID: \"2d82f463-3a4e-4344-b9cb-a4beaf44a0b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"
Jul 28 20:41:01.895056 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:01.894968    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"
Jul 28 20:41:02.030736 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:02.030711    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"]
Jul 28 20:41:02.033907 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:41:02.033878    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d82f463_3a4e_4344_b9cb_a4beaf44a0b4.slice/crio-d44c28fb69cbe10a142a52a67a59b8478135cb9a4fd0ce07be82d85a0947ca6e WatchSource:0}: Error finding container d44c28fb69cbe10a142a52a67a59b8478135cb9a4fd0ce07be82d85a0947ca6e: Status 404 returned error can't find the container with id d44c28fb69cbe10a142a52a67a59b8478135cb9a4fd0ce07be82d85a0947ca6e
Jul 28 20:41:02.315038 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:02.314992    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4" event={"ID":"2d82f463-3a4e-4344-b9cb-a4beaf44a0b4","Type":"ContainerStarted","Data":"d44c28fb69cbe10a142a52a67a59b8478135cb9a4fd0ce07be82d85a0947ca6e"}
Jul 28 20:41:03.321898 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:03.321868    2569 generic.go:356] "Generic (PLEG): container finished" podID="2d82f463-3a4e-4344-b9cb-a4beaf44a0b4" containerID="88c0f794bcf0b06bc53957ba6f01cb31d1f960428d161b34d512cc2eb1c52a0e" exitCode=0
Jul 28 20:41:03.322335 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:03.321958    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4" event={"ID":"2d82f463-3a4e-4344-b9cb-a4beaf44a0b4","Type":"ContainerDied","Data":"88c0f794bcf0b06bc53957ba6f01cb31d1f960428d161b34d512cc2eb1c52a0e"}
Jul 28 20:41:04.460303 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:04.460266    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"
Jul 28 20:41:04.565660 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:04.565635    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d82f463-3a4e-4344-b9cb-a4beaf44a0b4-maas-api-service-ca\") pod \"2d82f463-3a4e-4344-b9cb-a4beaf44a0b4\" (UID: \"2d82f463-3a4e-4344-b9cb-a4beaf44a0b4\") "
Jul 28 20:41:04.565946 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:04.565925    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d82f463-3a4e-4344-b9cb-a4beaf44a0b4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d82f463-3a4e-4344-b9cb-a4beaf44a0b4" (UID: "2d82f463-3a4e-4344-b9cb-a4beaf44a0b4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:41:04.666878 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:04.666806    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d82f463-3a4e-4344-b9cb-a4beaf44a0b4-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:41:05.334436 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:05.334395    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4" event={"ID":"2d82f463-3a4e-4344-b9cb-a4beaf44a0b4","Type":"ContainerDied","Data":"d44c28fb69cbe10a142a52a67a59b8478135cb9a4fd0ce07be82d85a0947ca6e"}
Jul 28 20:41:05.334617 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:05.334445    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d44c28fb69cbe10a142a52a67a59b8478135cb9a4fd0ce07be82d85a0947ca6e"
Jul 28 20:41:05.334617 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:05.334413    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"
Jul 28 20:41:11.645337 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:11.645303    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:12.349200 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:12.349169    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:12.479717 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:12.479683    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:12.510272 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:12.510242    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"]
Jul 28 20:41:12.513464 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:12.513440    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-579f3e-82wk4"]
Jul 28 20:41:12.697033 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:12.697005    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:13.004532 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:13.004446    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:13.366426 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:13.366353    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d82f463-3a4e-4344-b9cb-a4beaf44a0b4" path="/var/lib/kubelet/pods/2d82f463-3a4e-4344-b9cb-a4beaf44a0b4/volumes"
Jul 28 20:41:19.441165 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:19.441132    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:19.529179 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:19.529152    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:19.769077 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:19.769005    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:20.112673 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:20.112596    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:23.858231 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:23.858194    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"]
Jul 28 20:41:23.858612 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:23.858598    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d82f463-3a4e-4344-b9cb-a4beaf44a0b4" containerName="revoke-keys"
Jul 28 20:41:23.858668 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:23.858614    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d82f463-3a4e-4344-b9cb-a4beaf44a0b4" containerName="revoke-keys"
Jul 28 20:41:23.858705 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:23.858688    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d82f463-3a4e-4344-b9cb-a4beaf44a0b4" containerName="revoke-keys"
Jul 28 20:41:23.861864 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:23.861848    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"
Jul 28 20:41:23.864201 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:23.864172    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:41:23.864356 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:23.864204    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:41:23.869608 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:23.869587    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"]
Jul 28 20:41:24.035027 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:24.034998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e53fef2-7ad7-4af3-9cec-17f7a158d909-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq\" (UID: \"6e53fef2-7ad7-4af3-9cec-17f7a158d909\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"
Jul 28 20:41:24.136495 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:24.136427    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e53fef2-7ad7-4af3-9cec-17f7a158d909-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq\" (UID: \"6e53fef2-7ad7-4af3-9cec-17f7a158d909\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"
Jul 28 20:41:24.137004 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:24.136983    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e53fef2-7ad7-4af3-9cec-17f7a158d909-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq\" (UID: \"6e53fef2-7ad7-4af3-9cec-17f7a158d909\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"
Jul 28 20:41:24.173720 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:24.173698    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"
Jul 28 20:41:24.305164 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:24.305117    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"]
Jul 28 20:41:24.312305 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:41:24.312259    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e53fef2_7ad7_4af3_9cec_17f7a158d909.slice/crio-cd9571ef087c6edc692e7bad4a8448c2957b505d094604c02e3da6f67c6d4c66 WatchSource:0}: Error finding container cd9571ef087c6edc692e7bad4a8448c2957b505d094604c02e3da6f67c6d4c66: Status 404 returned error can't find the container with id cd9571ef087c6edc692e7bad4a8448c2957b505d094604c02e3da6f67c6d4c66
Jul 28 20:41:24.430461 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:24.430430    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq" event={"ID":"6e53fef2-7ad7-4af3-9cec-17f7a158d909","Type":"ContainerStarted","Data":"cd9571ef087c6edc692e7bad4a8448c2957b505d094604c02e3da6f67c6d4c66"}
Jul 28 20:41:25.438341 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:25.438307    2569 generic.go:356] "Generic (PLEG): container finished" podID="6e53fef2-7ad7-4af3-9cec-17f7a158d909" containerID="c807e2f1f839849ed90ed183847c3782bb48b6678e2faf2e83391b1acda51179" exitCode=0
Jul 28 20:41:25.438720 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:25.438395    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq" event={"ID":"6e53fef2-7ad7-4af3-9cec-17f7a158d909","Type":"ContainerDied","Data":"c807e2f1f839849ed90ed183847c3782bb48b6678e2faf2e83391b1acda51179"}
Jul 28 20:41:26.578412 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:26.578382    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"
Jul 28 20:41:26.758601 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:26.758520    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e53fef2-7ad7-4af3-9cec-17f7a158d909-maas-api-service-ca\") pod \"6e53fef2-7ad7-4af3-9cec-17f7a158d909\" (UID: \"6e53fef2-7ad7-4af3-9cec-17f7a158d909\") "
Jul 28 20:41:26.758876 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:26.758842    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e53fef2-7ad7-4af3-9cec-17f7a158d909-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6e53fef2-7ad7-4af3-9cec-17f7a158d909" (UID: "6e53fef2-7ad7-4af3-9cec-17f7a158d909"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:41:26.859351 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:26.859327    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e53fef2-7ad7-4af3-9cec-17f7a158d909-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:41:27.450608 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:27.450581    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"
Jul 28 20:41:27.450766 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:27.450625    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq" event={"ID":"6e53fef2-7ad7-4af3-9cec-17f7a158d909","Type":"ContainerDied","Data":"cd9571ef087c6edc692e7bad4a8448c2957b505d094604c02e3da6f67c6d4c66"}
Jul 28 20:41:27.450766 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:27.450692    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd9571ef087c6edc692e7bad4a8448c2957b505d094604c02e3da6f67c6d4c66"
Jul 28 20:41:39.915854 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:39.915816    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"]
Jul 28 20:41:39.919763 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:39.919739    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-92ba10-c2lqq"]
Jul 28 20:41:40.425552 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:40.425500    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:40.628807 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:40.628766    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:41.366574 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:41.366534    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e53fef2-7ad7-4af3-9cec-17f7a158d909" path="/var/lib/kubelet/pods/6e53fef2-7ad7-4af3-9cec-17f7a158d909/volumes"
Jul 28 20:41:45.313784 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:45.313743    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"]
Jul 28 20:41:45.405288 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:45.405259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:41:59.519368 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:59.519340    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:41:59.529565 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:41:59.529537    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5m4m9_9f6fc39a-c1a5-44f7-94a2-099eb992e02d/ovn-acl-logging/0.log"
Jul 28 20:42:00.333190 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.333148    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc" podUID="a38975b4-d3ca-45a4-9336-16f4fd075199" containerName="main" containerID="cri-o://f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44" gracePeriod=45
Jul 28 20:42:00.585974 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.585913    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:42:00.623408 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.623378    2569 generic.go:356] "Generic (PLEG): container finished" podID="a38975b4-d3ca-45a4-9336-16f4fd075199" containerID="f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44" exitCode=0
Jul 28 20:42:00.623543 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.623424    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"
Jul 28 20:42:00.623543 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.623454    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc" event={"ID":"a38975b4-d3ca-45a4-9336-16f4fd075199","Type":"ContainerDied","Data":"f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44"}
Jul 28 20:42:00.623543 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.623487    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc" event={"ID":"a38975b4-d3ca-45a4-9336-16f4fd075199","Type":"ContainerDied","Data":"3f3f89d04c61037372578cde6811764b3f98471e93dbe0c328aac52a0b6c86d5"}
Jul 28 20:42:00.623543 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.623503    2569 scope.go:117] "RemoveContainer" containerID="f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44"
Jul 28 20:42:00.632002 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.631985    2569 scope.go:117] "RemoveContainer" containerID="f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44"
Jul 28 20:42:00.632268 ip-10-0-134-138 kubenswrapper[2569]: E0728 20:42:00.632250    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44\": container with ID starting with f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44 not found: ID does not exist" containerID="f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44"
Jul 28 20:42:00.632371 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.632301    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44"} err="failed to get container status \"f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44\": rpc error: code = NotFound desc = could not find container \"f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44\": container with ID starting with f70b35ee565314e87ec8987cbe88f06b1e423a124c801d592f3caf9cc6adda44 not found: ID does not exist"
Jul 28 20:42:00.646736 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.646717    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a38975b4-d3ca-45a4-9336-16f4fd075199-tls-certs\") pod \"a38975b4-d3ca-45a4-9336-16f4fd075199\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") "
Jul 28 20:42:00.646825 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.646772    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-dshm\") pod \"a38975b4-d3ca-45a4-9336-16f4fd075199\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") "
Jul 28 20:42:00.646825 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.646799    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpdds\" (UniqueName: \"kubernetes.io/projected/a38975b4-d3ca-45a4-9336-16f4fd075199-kube-api-access-xpdds\") pod \"a38975b4-d3ca-45a4-9336-16f4fd075199\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") "
Jul 28 20:42:00.646906 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.646853    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-model-cache\") pod \"a38975b4-d3ca-45a4-9336-16f4fd075199\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") "
Jul 28 20:42:00.646906 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.646881    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-home\") pod \"a38975b4-d3ca-45a4-9336-16f4fd075199\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") "
Jul 28 20:42:00.646998 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.646934    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-tmp-dir\") pod \"a38975b4-d3ca-45a4-9336-16f4fd075199\" (UID: \"a38975b4-d3ca-45a4-9336-16f4fd075199\") "
Jul 28 20:42:00.647185 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.647103    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-model-cache" (OuterVolumeSpecName: "model-cache") pod "a38975b4-d3ca-45a4-9336-16f4fd075199" (UID: "a38975b4-d3ca-45a4-9336-16f4fd075199"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:42:00.647305 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.647179    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-home" (OuterVolumeSpecName: "home") pod "a38975b4-d3ca-45a4-9336-16f4fd075199" (UID: "a38975b4-d3ca-45a4-9336-16f4fd075199"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:42:00.647305 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.647249    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-model-cache\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:42:00.647417 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.647343    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a38975b4-d3ca-45a4-9336-16f4fd075199" (UID: "a38975b4-d3ca-45a4-9336-16f4fd075199"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:42:00.648949 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.648924    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a38975b4-d3ca-45a4-9336-16f4fd075199-kube-api-access-xpdds" (OuterVolumeSpecName: "kube-api-access-xpdds") pod "a38975b4-d3ca-45a4-9336-16f4fd075199" (UID: "a38975b4-d3ca-45a4-9336-16f4fd075199"). InnerVolumeSpecName "kube-api-access-xpdds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:42:00.649028 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.648961    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a38975b4-d3ca-45a4-9336-16f4fd075199-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a38975b4-d3ca-45a4-9336-16f4fd075199" (UID: "a38975b4-d3ca-45a4-9336-16f4fd075199"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:42:00.649143 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.649122    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-dshm" (OuterVolumeSpecName: "dshm") pod "a38975b4-d3ca-45a4-9336-16f4fd075199" (UID: "a38975b4-d3ca-45a4-9336-16f4fd075199"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:42:00.747647 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.747626    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-home\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:42:00.747729 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.747666    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-tmp-dir\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:42:00.747729 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.747686    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a38975b4-d3ca-45a4-9336-16f4fd075199-tls-certs\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:42:00.747729 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.747694    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a38975b4-d3ca-45a4-9336-16f4fd075199-dshm\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:42:00.747729 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.747702    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpdds\" (UniqueName: \"kubernetes.io/projected/a38975b4-d3ca-45a4-9336-16f4fd075199-kube-api-access-xpdds\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:42:00.950441 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.950417    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"]
Jul 28 20:42:00.956416 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:00.956391    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-92ba10/ipp-route-92ba10-kserve-959bf8695-v68zc"]
Jul 28 20:42:01.365835 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:01.365753    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a38975b4-d3ca-45a4-9336-16f4fd075199" path="/var/lib/kubelet/pods/a38975b4-d3ca-45a4-9336-16f4fd075199/volumes"
Jul 28 20:42:23.650333 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:23.650249    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:23.880890 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:23.880834    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:39.826799 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:39.826766    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:39.998831 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:39.998797    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:40.297392 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.297360    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:40.532697 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.532657    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"]
Jul 28 20:42:40.533264 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.533244    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e53fef2-7ad7-4af3-9cec-17f7a158d909" containerName="revoke-keys"
Jul 28 20:42:40.533399 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.533265    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e53fef2-7ad7-4af3-9cec-17f7a158d909" containerName="revoke-keys"
Jul 28 20:42:40.533399 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.533303    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a38975b4-d3ca-45a4-9336-16f4fd075199" containerName="main"
Jul 28 20:42:40.533399 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.533313    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a38975b4-d3ca-45a4-9336-16f4fd075199" containerName="main"
Jul 28 20:42:40.533558 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.533410    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a38975b4-d3ca-45a4-9336-16f4fd075199" containerName="main"
Jul 28 20:42:40.533558 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.533423    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e53fef2-7ad7-4af3-9cec-17f7a158d909" containerName="revoke-keys"
Jul 28 20:42:40.536921 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.536899    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"
Jul 28 20:42:40.539586 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.539555    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:42:40.539922 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.539902    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:42:40.560389 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.560323    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:40.561059 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.561034    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"]
Jul 28 20:42:40.565131 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.565106    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f63bd3-1488-446c-827d-4eef25d3ec58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-37887f-48psc\" (UID: \"c0f63bd3-1488-446c-827d-4eef25d3ec58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"
Jul 28 20:42:40.665626 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.665595    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f63bd3-1488-446c-827d-4eef25d3ec58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-37887f-48psc\" (UID: \"c0f63bd3-1488-446c-827d-4eef25d3ec58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"
Jul 28 20:42:40.666445 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.666422    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f63bd3-1488-446c-827d-4eef25d3ec58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-37887f-48psc\" (UID: \"c0f63bd3-1488-446c-827d-4eef25d3ec58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"
Jul 28 20:42:40.848798 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.848715    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"
Jul 28 20:42:40.996519 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:40.996491    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"]
Jul 28 20:42:40.997634 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:42:40.997609    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f63bd3_1488_446c_827d_4eef25d3ec58.slice/crio-dfc2356479f0becffb3e5e54b2517fc4a67570ff21bc72ce10deb0f3e7e5e72f WatchSource:0}: Error finding container dfc2356479f0becffb3e5e54b2517fc4a67570ff21bc72ce10deb0f3e7e5e72f: Status 404 returned error can't find the container with id dfc2356479f0becffb3e5e54b2517fc4a67570ff21bc72ce10deb0f3e7e5e72f
Jul 28 20:42:41.833810 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:41.833774    2569 generic.go:356] "Generic (PLEG): container finished" podID="c0f63bd3-1488-446c-827d-4eef25d3ec58" containerID="76be897b9cae8e4dee178e3da6945548dcf602b8334e03c0ced2e5bac4da268e" exitCode=0
Jul 28 20:42:41.833946 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:41.833834    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc" event={"ID":"c0f63bd3-1488-446c-827d-4eef25d3ec58","Type":"ContainerDied","Data":"76be897b9cae8e4dee178e3da6945548dcf602b8334e03c0ced2e5bac4da268e"}
Jul 28 20:42:41.833946 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:41.833894    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc" event={"ID":"c0f63bd3-1488-446c-827d-4eef25d3ec58","Type":"ContainerStarted","Data":"dfc2356479f0becffb3e5e54b2517fc4a67570ff21bc72ce10deb0f3e7e5e72f"}
Jul 28 20:42:42.969508 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:42.969486    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"
Jul 28 20:42:42.985676 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:42.985656    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f63bd3-1488-446c-827d-4eef25d3ec58-maas-api-service-ca\") pod \"c0f63bd3-1488-446c-827d-4eef25d3ec58\" (UID: \"c0f63bd3-1488-446c-827d-4eef25d3ec58\") "
Jul 28 20:42:42.985955 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:42.985935    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f63bd3-1488-446c-827d-4eef25d3ec58-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c0f63bd3-1488-446c-827d-4eef25d3ec58" (UID: "c0f63bd3-1488-446c-827d-4eef25d3ec58"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:42:43.087043 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:43.087010    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f63bd3-1488-446c-827d-4eef25d3ec58-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:42:43.845841 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:43.845810    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"
Jul 28 20:42:43.846005 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:43.845840    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc" event={"ID":"c0f63bd3-1488-446c-827d-4eef25d3ec58","Type":"ContainerDied","Data":"dfc2356479f0becffb3e5e54b2517fc4a67570ff21bc72ce10deb0f3e7e5e72f"}
Jul 28 20:42:43.846005 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:43.845894    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dfc2356479f0becffb3e5e54b2517fc4a67570ff21bc72ce10deb0f3e7e5e72f"
Jul 28 20:42:46.177732 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:46.177697    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:46.388588 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:46.388545    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:46.578426 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:46.578341    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:46.665732 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:46.665694    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"]
Jul 28 20:42:46.683392 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:46.683362    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-37887f-48psc"]
Jul 28 20:42:46.975976 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:46.975912    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:47.309297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:47.309196    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:47.365679 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:47.365650    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0f63bd3-1488-446c-827d-4eef25d3ec58" path="/var/lib/kubelet/pods/c0f63bd3-1488-446c-827d-4eef25d3ec58/volumes"
Jul 28 20:42:54.483905 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:54.483869    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:54.686231 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:54.686201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:54.894774 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:54.894696    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:55.094552 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.094515    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:55.208372 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.208342    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"]
Jul 28 20:42:55.208748 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.208735    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f63bd3-1488-446c-827d-4eef25d3ec58" containerName="revoke-keys"
Jul 28 20:42:55.208793 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.208749    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f63bd3-1488-446c-827d-4eef25d3ec58" containerName="revoke-keys"
Jul 28 20:42:55.208829 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.208807    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f63bd3-1488-446c-827d-4eef25d3ec58" containerName="revoke-keys"
Jul 28 20:42:55.211976 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.211947    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"
Jul 28 20:42:55.214456 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.214434    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:42:55.214655 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.214642    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:42:55.235394 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.235370    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"]
Jul 28 20:42:55.301532 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.301509    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c58a14-5eb4-4a51-8345-03f54d9967c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd\" (UID: \"78c58a14-5eb4-4a51-8345-03f54d9967c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"
Jul 28 20:42:55.401924 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.401898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c58a14-5eb4-4a51-8345-03f54d9967c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd\" (UID: \"78c58a14-5eb4-4a51-8345-03f54d9967c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"
Jul 28 20:42:55.402505 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.402482    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c58a14-5eb4-4a51-8345-03f54d9967c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd\" (UID: \"78c58a14-5eb4-4a51-8345-03f54d9967c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"
Jul 28 20:42:55.522411 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.522357    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"
Jul 28 20:42:55.685309 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.685259    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"]
Jul 28 20:42:55.688485 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:42:55.688456    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c58a14_5eb4_4a51_8345_03f54d9967c2.slice/crio-0c9bd8b6d8e80837425fe63805fe99c09252f4e3e86286ea98e151a88e8cb721 WatchSource:0}: Error finding container 0c9bd8b6d8e80837425fe63805fe99c09252f4e3e86286ea98e151a88e8cb721: Status 404 returned error can't find the container with id 0c9bd8b6d8e80837425fe63805fe99c09252f4e3e86286ea98e151a88e8cb721
Jul 28 20:42:55.908177 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:55.908087    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd" event={"ID":"78c58a14-5eb4-4a51-8345-03f54d9967c2","Type":"ContainerStarted","Data":"0c9bd8b6d8e80837425fe63805fe99c09252f4e3e86286ea98e151a88e8cb721"}
Jul 28 20:42:56.914686 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:56.914597    2569 generic.go:356] "Generic (PLEG): container finished" podID="78c58a14-5eb4-4a51-8345-03f54d9967c2" containerID="c6e5feba44665d56204f8a1ab9cdd1567908f99ca61b4d235149f58cf4092597" exitCode=0
Jul 28 20:42:56.915120 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:56.914684    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd" event={"ID":"78c58a14-5eb4-4a51-8345-03f54d9967c2","Type":"ContainerDied","Data":"c6e5feba44665d56204f8a1ab9cdd1567908f99ca61b4d235149f58cf4092597"}
Jul 28 20:42:58.050445 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:58.050424    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"
Jul 28 20:42:58.128722 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:58.128695    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c58a14-5eb4-4a51-8345-03f54d9967c2-maas-api-service-ca\") pod \"78c58a14-5eb4-4a51-8345-03f54d9967c2\" (UID: \"78c58a14-5eb4-4a51-8345-03f54d9967c2\") "
Jul 28 20:42:58.129025 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:58.129003    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78c58a14-5eb4-4a51-8345-03f54d9967c2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "78c58a14-5eb4-4a51-8345-03f54d9967c2" (UID: "78c58a14-5eb4-4a51-8345-03f54d9967c2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:42:58.230437 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:58.230413    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78c58a14-5eb4-4a51-8345-03f54d9967c2-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:42:58.926137 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:58.926103    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"
Jul 28 20:42:58.926137 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:58.926113    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd" event={"ID":"78c58a14-5eb4-4a51-8345-03f54d9967c2","Type":"ContainerDied","Data":"0c9bd8b6d8e80837425fe63805fe99c09252f4e3e86286ea98e151a88e8cb721"}
Jul 28 20:42:58.926357 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:42:58.926153    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c9bd8b6d8e80837425fe63805fe99c09252f4e3e86286ea98e151a88e8cb721"
Jul 28 20:43:06.173044 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:06.173003    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:06.196486 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:06.196460    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l"]
Jul 28 20:43:06.213272 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:06.213242    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed54db-29754510-lwp4l"]
Jul 28 20:43:06.511025 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:06.510936    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:06.710134 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:06.710104    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"]
Jul 28 20:43:06.720586 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:06.720560    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed54db-dkrkd"]
Jul 28 20:43:06.726782 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:06.726758    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:06.877571 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:06.877504    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:07.152365 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:07.152274    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:07.366098 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:07.366066    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f451faa-6e75-48b4-be4a-cefe3327b9bb" path="/var/lib/kubelet/pods/5f451faa-6e75-48b4-be4a-cefe3327b9bb/volumes"
Jul 28 20:43:07.366471 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:07.366404    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78c58a14-5eb4-4a51-8345-03f54d9967c2" path="/var/lib/kubelet/pods/78c58a14-5eb4-4a51-8345-03f54d9967c2/volumes"
Jul 28 20:43:13.828405 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:13.828373    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:13.928246 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:13.928218    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:14.204228 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.204194    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:14.384020 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.383988    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:14.499425 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.499349    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"]
Jul 28 20:43:14.499752 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.499739    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78c58a14-5eb4-4a51-8345-03f54d9967c2" containerName="revoke-keys"
Jul 28 20:43:14.499805 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.499754    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="78c58a14-5eb4-4a51-8345-03f54d9967c2" containerName="revoke-keys"
Jul 28 20:43:14.499856 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.499830    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="78c58a14-5eb4-4a51-8345-03f54d9967c2" containerName="revoke-keys"
Jul 28 20:43:14.502689 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.502669    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"
Jul 28 20:43:14.505093 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.505077    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9zkwl\""
Jul 28 20:43:14.505482 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.505465    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:43:14.513188 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.513165    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"]
Jul 28 20:43:14.675641 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.675611    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb1099a5-a1a2-43fd-af50-fba89cfa7278-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt\" (UID: \"eb1099a5-a1a2-43fd-af50-fba89cfa7278\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"
Jul 28 20:43:14.776274 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.776193    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb1099a5-a1a2-43fd-af50-fba89cfa7278-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt\" (UID: \"eb1099a5-a1a2-43fd-af50-fba89cfa7278\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"
Jul 28 20:43:14.776816 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.776797    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb1099a5-a1a2-43fd-af50-fba89cfa7278-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt\" (UID: \"eb1099a5-a1a2-43fd-af50-fba89cfa7278\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"
Jul 28 20:43:14.813224 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.813196    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"
Jul 28 20:43:14.952790 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:14.952763    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"]
Jul 28 20:43:14.955334 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:43:14.955306    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb1099a5_a1a2_43fd_af50_fba89cfa7278.slice/crio-12835711af8ef6d7467a15142aeec7102142eb4ebad217f0fa51be1dff329945 WatchSource:0}: Error finding container 12835711af8ef6d7467a15142aeec7102142eb4ebad217f0fa51be1dff329945: Status 404 returned error can't find the container with id 12835711af8ef6d7467a15142aeec7102142eb4ebad217f0fa51be1dff329945
Jul 28 20:43:15.009744 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:15.009720    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt" event={"ID":"eb1099a5-a1a2-43fd-af50-fba89cfa7278","Type":"ContainerStarted","Data":"12835711af8ef6d7467a15142aeec7102142eb4ebad217f0fa51be1dff329945"}
Jul 28 20:43:16.016754 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:16.016727    2569 generic.go:356] "Generic (PLEG): container finished" podID="eb1099a5-a1a2-43fd-af50-fba89cfa7278" containerID="6ae77248224712947fa531f8bd4ef4476fc313c00e8f14fce18565ca92d25822" exitCode=0
Jul 28 20:43:16.017132 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:16.016764    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt" event={"ID":"eb1099a5-a1a2-43fd-af50-fba89cfa7278","Type":"ContainerDied","Data":"6ae77248224712947fa531f8bd4ef4476fc313c00e8f14fce18565ca92d25822"}
Jul 28 20:43:17.149868 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:17.149843    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"
Jul 28 20:43:17.297322 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:17.297245    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb1099a5-a1a2-43fd-af50-fba89cfa7278-maas-api-service-ca\") pod \"eb1099a5-a1a2-43fd-af50-fba89cfa7278\" (UID: \"eb1099a5-a1a2-43fd-af50-fba89cfa7278\") "
Jul 28 20:43:17.297546 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:17.297523    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb1099a5-a1a2-43fd-af50-fba89cfa7278-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb1099a5-a1a2-43fd-af50-fba89cfa7278" (UID: "eb1099a5-a1a2-43fd-af50-fba89cfa7278"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:43:17.398267 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:17.398243    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb1099a5-a1a2-43fd-af50-fba89cfa7278-maas-api-service-ca\") on node \"ip-10-0-134-138.ec2.internal\" DevicePath \"\""
Jul 28 20:43:18.029273 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:18.029245    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"
Jul 28 20:43:18.029461 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:18.029302    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt" event={"ID":"eb1099a5-a1a2-43fd-af50-fba89cfa7278","Type":"ContainerDied","Data":"12835711af8ef6d7467a15142aeec7102142eb4ebad217f0fa51be1dff329945"}
Jul 28 20:43:18.029461 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:18.029351    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12835711af8ef6d7467a15142aeec7102142eb4ebad217f0fa51be1dff329945"
Jul 28 20:43:25.266271 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:25.266237    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc"]
Jul 28 20:43:25.272261 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:25.272228    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1231ba-29754510-8vzpc"]
Jul 28 20:43:25.317702 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:25.317676    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:25.367180 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:25.367147    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b823a9a-7579-4d2a-af69-f47c15805cb6" path="/var/lib/kubelet/pods/4b823a9a-7579-4d2a-af69-f47c15805cb6/volumes"
Jul 28 20:43:25.438441 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:25.438409    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:25.602452 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:25.602386    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"]
Jul 28 20:43:25.608883 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:25.608861    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1231ba-g82kt"]
Jul 28 20:43:25.625722 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:25.625694    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:26.019271 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:26.019245    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:27.366165 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:27.366133    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb1099a5-a1a2-43fd-af50-fba89cfa7278" path="/var/lib/kubelet/pods/eb1099a5-a1a2-43fd-af50-fba89cfa7278/volumes"
Jul 28 20:43:33.825572 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:33.825540    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:33.982363 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:33.982329    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:34.189191 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:34.189163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:34.409686 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:34.409654    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:40.859803 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:40.859772    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ddmnl_8d38055c-6d25-4719-b1ce-b49781a2c73c/manager/0.log"
Jul 28 20:43:40.980341 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:40.980302    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b5686dc45-mjwfk_90019d4c-c698-4d73-b8b9-f64c8f914318/manager/0.log"
Jul 28 20:43:41.494269 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:41.494243    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85967bc8b9-ddpnf_c48574d1-ce18-4c3c-9b12-7828cc8f5df0/copy-manifests/0.log"
Jul 28 20:43:41.509658 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:41.509634    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85967bc8b9-ddpnf_c48574d1-ce18-4c3c-9b12-7828cc8f5df0/manager/0.log"
Jul 28 20:43:42.149590 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:42.149551    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:42.259257 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:42.259209    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:42.361911 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:42.361887    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754510-mnbx8_296f127e-f334-4a65-8b60-bb34285e57e7/cleanup/0.log"
Jul 28 20:43:44.481734 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:44.481705    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2g5sw_4384066d-b1a6-43f1-a184-cea1f130eb1c/manager/0.log"
Jul 28 20:43:45.348041 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:45.348014    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-579d8dfb47-bbhf8_281eb0aa-e5cf-4071-a6bf-de2fa275ed85/kube-auth-proxy/0.log"
Jul 28 20:43:47.458260 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:47.458230    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-w6g5p_932bb7be-5749-49be-9779-bf91092722f7/main/0.log"
Jul 28 20:43:47.861654 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:47.861567    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft2vt2_1f2756a3-16e9-4d20-9782-2f49e4e728dc/main/0.log"
Jul 28 20:43:54.273996 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.273916    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bz499/must-gather-64bch"]
Jul 28 20:43:54.274404 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.274313    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb1099a5-a1a2-43fd-af50-fba89cfa7278" containerName="revoke-keys"
Jul 28 20:43:54.274404 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.274325    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb1099a5-a1a2-43fd-af50-fba89cfa7278" containerName="revoke-keys"
Jul 28 20:43:54.274404 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.274401    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb1099a5-a1a2-43fd-af50-fba89cfa7278" containerName="revoke-keys"
Jul 28 20:43:54.277581 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.277561    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bz499/must-gather-64bch"
Jul 28 20:43:54.283459 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.283429    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bz499\"/\"default-dockercfg-mdklh\""
Jul 28 20:43:54.283936 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.283884    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bz499\"/\"openshift-service-ca.crt\""
Jul 28 20:43:54.284153 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.284134    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bz499\"/\"kube-root-ca.crt\""
Jul 28 20:43:54.315475 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.315446    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:54.365934 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.365912    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bz499/must-gather-64bch"]
Jul 28 20:43:54.405262 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.405238    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/79446444-b98a-4b2b-bc35-8d55bf449bc0-must-gather-output\") pod \"must-gather-64bch\" (UID: \"79446444-b98a-4b2b-bc35-8d55bf449bc0\") " pod="openshift-must-gather-bz499/must-gather-64bch"
Jul 28 20:43:54.405385 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.405265    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc67k\" (UniqueName: \"kubernetes.io/projected/79446444-b98a-4b2b-bc35-8d55bf449bc0-kube-api-access-kc67k\") pod \"must-gather-64bch\" (UID: \"79446444-b98a-4b2b-bc35-8d55bf449bc0\") " pod="openshift-must-gather-bz499/must-gather-64bch"
Jul 28 20:43:54.465826 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.465800    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:54.506478 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.506455    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/79446444-b98a-4b2b-bc35-8d55bf449bc0-must-gather-output\") pod \"must-gather-64bch\" (UID: \"79446444-b98a-4b2b-bc35-8d55bf449bc0\") " pod="openshift-must-gather-bz499/must-gather-64bch"
Jul 28 20:43:54.506603 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.506488    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc67k\" (UniqueName: \"kubernetes.io/projected/79446444-b98a-4b2b-bc35-8d55bf449bc0-kube-api-access-kc67k\") pod \"must-gather-64bch\" (UID: \"79446444-b98a-4b2b-bc35-8d55bf449bc0\") " pod="openshift-must-gather-bz499/must-gather-64bch"
Jul 28 20:43:54.506772 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.506752    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/79446444-b98a-4b2b-bc35-8d55bf449bc0-must-gather-output\") pod \"must-gather-64bch\" (UID: \"79446444-b98a-4b2b-bc35-8d55bf449bc0\") " pod="openshift-must-gather-bz499/must-gather-64bch"
Jul 28 20:43:54.516249 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.516226    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc67k\" (UniqueName: \"kubernetes.io/projected/79446444-b98a-4b2b-bc35-8d55bf449bc0-kube-api-access-kc67k\") pod \"must-gather-64bch\" (UID: \"79446444-b98a-4b2b-bc35-8d55bf449bc0\") " pod="openshift-must-gather-bz499/must-gather-64bch"
Jul 28 20:43:54.586297 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.586230    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bz499/must-gather-64bch"
Jul 28 20:43:54.738568 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.738542    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bz499/must-gather-64bch"]
Jul 28 20:43:54.741057 ip-10-0-134-138 kubenswrapper[2569]: W0728 20:43:54.741031    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79446444_b98a_4b2b_bc35_8d55bf449bc0.slice/crio-f0c455a9a016d5ed95b5e6e46cd0acfaa8df909ade19a79923cfa37d6a8ab787 WatchSource:0}: Error finding container f0c455a9a016d5ed95b5e6e46cd0acfaa8df909ade19a79923cfa37d6a8ab787: Status 404 returned error can't find the container with id f0c455a9a016d5ed95b5e6e46cd0acfaa8df909ade19a79923cfa37d6a8ab787
Jul 28 20:43:54.742950 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:54.742929    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:43:55.217629 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:55.217594    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bz499/must-gather-64bch" event={"ID":"79446444-b98a-4b2b-bc35-8d55bf449bc0","Type":"ContainerStarted","Data":"f0c455a9a016d5ed95b5e6e46cd0acfaa8df909ade19a79923cfa37d6a8ab787"}
Jul 28 20:43:56.691527 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:56.691479    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:59.755615 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:59.755500    2569 scope.go:117] "RemoveContainer" containerID="7e2619cb507ab41ffde0793e74c81202dc2843a8c079602ed195e95dd5a739df"
Jul 28 20:43:59.766378 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:43:59.764033    2569 scope.go:117] "RemoveContainer" containerID="de83bad86cb4619beab68085e66bf1aac904e3398f16bd94828a3376e1e8d990"
Jul 28 20:44:14.703056 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:14.703019    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:44:14.804530 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:14.804499    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:44:17.080256 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:17.080198    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log" limit=1024
Jul 28 20:44:22.365381 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:22.365310    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bz499/must-gather-64bch" event={"ID":"79446444-b98a-4b2b-bc35-8d55bf449bc0","Type":"ContainerStarted","Data":"c3d540254c427e3c56e6a1ac63dfa97af542865ed2b2e657ed61ec4e1b48a0eb"}
Jul 28 20:44:22.365381 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:22.365362    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bz499/must-gather-64bch" event={"ID":"79446444-b98a-4b2b-bc35-8d55bf449bc0","Type":"ContainerStarted","Data":"5a395fe0c722f3358ac1e96bac4a561c4f4910223394c5937df6f7e6f0b96c74"}
Jul 28 20:44:22.384011 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:22.383957    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bz499/must-gather-64bch" podStartSLOduration=1.438955387 podStartE2EDuration="28.383938178s" totalImagesPullingTime="26.944982791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:43:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:44:22.382205168 +0000 UTC m=+3443.562722226" watchObservedRunningTime="2026-07-28 20:44:22.383938178 +0000 UTC m=+3443.564455235"
Jul 28 20:44:23.354602 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:23.354572    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hcb5b_4028a654-3327-4719-b4d0-6434b1573bbd/global-pull-secret-syncer/0.log"
Jul 28 20:44:23.481932 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:23.481902    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gfhzh_3b75f10f-a52b-469e-a350-6e750c24fbc0/konnectivity-agent/0.log"
Jul 28 20:44:23.636609 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:23.636537    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-138.ec2.internal_f39b57f429370ddf00b5bf9b9f3d71ea/haproxy/0.log"
Jul 28 20:44:29.090089 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:29.089998    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2g5sw_4384066d-b1a6-43f1-a184-cea1f130eb1c/manager/0.log"
Jul 28 20:44:30.822112 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:30.822049    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_433ab26f-8e8f-438e-afdd-328326afce93/alertmanager/0.log"
Jul 28 20:44:30.863907 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:30.863867    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_433ab26f-8e8f-438e-afdd-328326afce93/config-reloader/0.log"
Jul 28 20:44:30.896568 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:30.896460    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_433ab26f-8e8f-438e-afdd-328326afce93/kube-rbac-proxy-web/0.log"
Jul 28 20:44:30.931266 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:30.931240    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_433ab26f-8e8f-438e-afdd-328326afce93/kube-rbac-proxy/0.log"
Jul 28 20:44:30.968742 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:30.968710    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_433ab26f-8e8f-438e-afdd-328326afce93/kube-rbac-proxy-metric/0.log"
Jul 28 20:44:31.006222 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.006147    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_433ab26f-8e8f-438e-afdd-328326afce93/prom-label-proxy/0.log"
Jul 28 20:44:31.035644 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.035594    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_433ab26f-8e8f-438e-afdd-328326afce93/init-config-reloader/0.log"
Jul 28 20:44:31.119595 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.119531    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-956jc_973f742f-b3da-44ee-b805-af2d1db8fdf0/kube-state-metrics/0.log"
Jul 28 20:44:31.149118 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.149081    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-956jc_973f742f-b3da-44ee-b805-af2d1db8fdf0/kube-rbac-proxy-main/0.log"
Jul 28 20:44:31.186238 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.186204    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-956jc_973f742f-b3da-44ee-b805-af2d1db8fdf0/kube-rbac-proxy-self/0.log"
Jul 28 20:44:31.225201 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.225170    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5b7c949b84-q66pt_33e28913-f966-4ddb-93ac-d9887d8ea767/metrics-server/0.log"
Jul 28 20:44:31.259392 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.259310    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-rm5t5_fa37ab8f-afd8-4062-bf21-5cc435623849/monitoring-plugin/0.log"
Jul 28 20:44:31.400676 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.400648    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lbz5l_6149ebba-f436-44a9-851d-b9e6f4c543bf/node-exporter/0.log"
Jul 28 20:44:31.433069 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.433042    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lbz5l_6149ebba-f436-44a9-851d-b9e6f4c543bf/kube-rbac-proxy/0.log"
Jul 28 20:44:31.455620 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.455520    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lbz5l_6149ebba-f436-44a9-851d-b9e6f4c543bf/init-textfile/0.log"
Jul 28 20:44:31.576145 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.576061    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-5v5lr_4a983224-41c0-4cb5-974b-bc589225ca41/kube-rbac-proxy-main/0.log"
Jul 28 20:44:31.599685 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.599613    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-5v5lr_4a983224-41c0-4cb5-974b-bc589225ca41/kube-rbac-proxy-self/0.log"
Jul 28 20:44:31.623144 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.623109    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-5v5lr_4a983224-41c0-4cb5-974b-bc589225ca41/openshift-state-metrics/0.log"
Jul 28 20:44:31.668169 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.668140    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0d43a27-65c9-4966-83c2-dd85dd344c88/prometheus/0.log"
Jul 28 20:44:31.692033 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.691994    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0d43a27-65c9-4966-83c2-dd85dd344c88/config-reloader/0.log"
Jul 28 20:44:31.718324 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.718265    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0d43a27-65c9-4966-83c2-dd85dd344c88/thanos-sidecar/0.log"
Jul 28 20:44:31.749617 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.749587    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0d43a27-65c9-4966-83c2-dd85dd344c88/kube-rbac-proxy-web/0.log"
Jul 28 20:44:31.779851 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.779824    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0d43a27-65c9-4966-83c2-dd85dd344c88/kube-rbac-proxy/0.log"
Jul 28 20:44:31.811541 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.811508    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0d43a27-65c9-4966-83c2-dd85dd344c88/kube-rbac-proxy-thanos/0.log"
Jul 28 20:44:31.845031 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.844950    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d0d43a27-65c9-4966-83c2-dd85dd344c88/init-config-reloader/0.log"
Jul 28 20:44:31.880931 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.880901    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-j2qdz_49041c8a-764a-41ee-aa27-96ff41625896/prometheus-operator/0.log"
Jul 28 20:44:31.908039 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.908015    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-j2qdz_49041c8a-764a-41ee-aa27-96ff41625896/kube-rbac-proxy/0.log"
Jul 28 20:44:31.936442 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.936417    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-9rcqg_c6ed8149-d7bf-49b0-b7b2-f165f1ebd63d/prometheus-operator-admission-webhook/0.log"
Jul 28 20:44:31.970837 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.970809    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-769445799f-6j6h5_8841f1be-3558-4ac1-bb6d-bf3e38f2b51f/telemeter-client/0.log"
Jul 28 20:44:31.999163 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:31.999106    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-769445799f-6j6h5_8841f1be-3558-4ac1-bb6d-bf3e38f2b51f/reload/0.log"
Jul 28 20:44:32.039790 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:32.039718    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-769445799f-6j6h5_8841f1be-3558-4ac1-bb6d-bf3e38f2b51f/kube-rbac-proxy/0.log"
Jul 28 20:44:32.078985 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:32.078954    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b77bc96c6-z2s7q_8b4515b9-2bf8-4567-8a34-d90f74e2de2c/thanos-query/0.log"
Jul 28 20:44:32.104345 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:32.104251    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b77bc96c6-z2s7q_8b4515b9-2bf8-4567-8a34-d90f74e2de2c/kube-rbac-proxy-web/0.log"
Jul 28 20:44:32.132012 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:32.131983    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b77bc96c6-z2s7q_8b4515b9-2bf8-4567-8a34-d90f74e2de2c/kube-rbac-proxy/0.log"
Jul 28 20:44:32.164793 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:32.164764    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b77bc96c6-z2s7q_8b4515b9-2bf8-4567-8a34-d90f74e2de2c/prom-label-proxy/0.log"
Jul 28 20:44:32.190647 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:32.190618    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b77bc96c6-z2s7q_8b4515b9-2bf8-4567-8a34-d90f74e2de2c/kube-rbac-proxy-rules/0.log"
Jul 28 20:44:32.214177 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:32.214147    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b77bc96c6-z2s7q_8b4515b9-2bf8-4567-8a34-d90f74e2de2c/kube-rbac-proxy-metrics/0.log"
Jul 28 20:44:33.046240 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.046203    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"]
Jul 28 20:44:33.051354 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.051327    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.065347 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.065316    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"]
Jul 28 20:44:33.170781 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.170749    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-sys\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.170973 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.170806    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-podres\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.170973 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.170840    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-lib-modules\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.170973 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.170863    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt8gf\" (UniqueName: \"kubernetes.io/projected/878dd51a-f75f-4885-ae22-3c32bdcbf299-kube-api-access-gt8gf\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.170973 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.170959    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-proc\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.272343 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.272309    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-proc\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.272503 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.272371    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-sys\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.272503 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.272429    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-proc\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.272503 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.272439    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-podres\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.272503 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.272497    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-podres\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.272666 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.272521    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-sys\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.272666 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.272531    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-lib-modules\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.272666 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.272590    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt8gf\" (UniqueName: \"kubernetes.io/projected/878dd51a-f75f-4885-ae22-3c32bdcbf299-kube-api-access-gt8gf\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.272666 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.272649    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/878dd51a-f75f-4885-ae22-3c32bdcbf299-lib-modules\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.282387 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.282349    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt8gf\" (UniqueName: \"kubernetes.io/projected/878dd51a-f75f-4885-ae22-3c32bdcbf299-kube-api-access-gt8gf\") pod \"perf-node-gather-daemonset-fdbt5\" (UID: \"878dd51a-f75f-4885-ae22-3c32bdcbf299\") " pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.367630 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.367547    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:33.556407 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:33.556373    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"]
Jul 28 20:44:34.459720 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:34.459664    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5" event={"ID":"878dd51a-f75f-4885-ae22-3c32bdcbf299","Type":"ContainerStarted","Data":"ee15b948d5e12f6c64943e20f8dd1e1eb1bd1aa9b5a4de2f5487c3622d88a011"}
Jul 28 20:44:34.460232 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:34.459741    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5" event={"ID":"878dd51a-f75f-4885-ae22-3c32bdcbf299","Type":"ContainerStarted","Data":"c796e01b0d78c1b22c609e36dd40cc90331c6037c474e8794de2a39fa825e8af"}
Jul 28 20:44:34.460568 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:34.460545    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:34.483596 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:34.483536    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5" podStartSLOduration=1.483517479 podStartE2EDuration="1.483517479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:44:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:44:34.480779801 +0000 UTC m=+3455.661296858" watchObservedRunningTime="2026-07-28 20:44:34.483517479 +0000 UTC m=+3455.664034539"
Jul 28 20:44:34.585004 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:34.584978    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7c7794ffb5-jt8bp_08e3d059-8f02-43f7-af82-3fe2345a6cfa/console/0.log"
Jul 28 20:44:36.802690 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:36.802653    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xlrkr_838e134c-af5c-4c02-bfc5-886832884a5c/dns/0.log"
Jul 28 20:44:36.876389 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:36.876362    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xlrkr_838e134c-af5c-4c02-bfc5-886832884a5c/kube-rbac-proxy/0.log"
Jul 28 20:44:36.968105 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:36.968075    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2m75c_aee74d7d-4f10-43b5-b805-90cd335b79fd/dns-node-resolver/0.log"
Jul 28 20:44:37.775988 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:37.775938    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-swxbg_9ad78daf-e4fe-4ef6-b833-308bd3211ea3/node-ca/0.log"
Jul 28 20:44:39.086428 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:39.086398    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-579d8dfb47-bbhf8_281eb0aa-e5cf-4071-a6bf-de2fa275ed85/kube-auth-proxy/0.log"
Jul 28 20:44:40.215412 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:40.215380    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nwvlj_f3fd2311-f24e-41a4-a01e-f6f388dec0ae/serve-healthcheck-canary/0.log"
Jul 28 20:44:40.994815 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:40.994784    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nt9qg_58e5525f-07b2-4562-b3bd-452985cbba23/kube-rbac-proxy/0.log"
Jul 28 20:44:41.019901 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:41.019871    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nt9qg_58e5525f-07b2-4562-b3bd-452985cbba23/exporter/0.log"
Jul 28 20:44:41.044742 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:41.044709    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nt9qg_58e5525f-07b2-4562-b3bd-452985cbba23/extractor/0.log"
Jul 28 20:44:41.484302 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:41.484249    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bz499/perf-node-gather-daemonset-fdbt5"
Jul 28 20:44:43.345267 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:43.345224    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-579d8dfb47-bbhf8_281eb0aa-e5cf-4071-a6bf-de2fa275ed85/kube-auth-proxy/0.log"
Jul 28 20:44:43.810608 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:43.810570    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ddmnl_8d38055c-6d25-4719-b1ce-b49781a2c73c/manager/0.log"
Jul 28 20:44:43.987504 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:43.987466    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b5686dc45-mjwfk_90019d4c-c698-4d73-b8b9-f64c8f914318/manager/0.log"
Jul 28 20:44:44.391868 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:44.391825    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85967bc8b9-ddpnf_c48574d1-ce18-4c3c-9b12-7828cc8f5df0/manager/0.log"
Jul 28 20:44:44.432847 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:44.432819    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85967bc8b9-ddpnf_c48574d1-ce18-4c3c-9b12-7828cc8f5df0/copy-manifests/0.log"
Jul 28 20:44:45.151473 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:45.151442    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-579d8dfb47-bbhf8_281eb0aa-e5cf-4071-a6bf-de2fa275ed85/kube-auth-proxy/0.log"
Jul 28 20:44:45.920152 ip-10-0-134-138 kubenswrapper[2569]: I0728 20:44:45.920121    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6846c7dcc9-58phh_919dcb98-c69d-42f8-8169-b761654a6b3a/manager/0.log"